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SECTION  ONE 


INTRODUCTION 


The  issue  of  how  to  define  “situation  awareness”--  let  alone  how  to  measure  it  -  has  perplexed 
the  scientific  and  operational  communities  for  many  years.  In  fact.  General  Merrill  A.  McPeak, 
former  USAF  Chief  of  Staff,  has  stated  “...I  know  it  when  1  see  it...”  but  “Can  it  be  measured 
objectively?. .Can  it  be  learned?”  (letter,  17  July,  1991,  SG/XO).  This  interest  at  the  highest 
levels  of  command  stems  from  the  growing  belief  that  situation  awareness  (SA)  is  a  major 
contributor  to  both  mission  failure  and  accidents.  Endsley  ( 1 988)  cited  SA  as  the  most  important 
factor  in  improving  mission  effectiveness.  Hartell,  Smith,  and  Prince  (1991)  reported  that  a  lack 
of  SA  was  given  most  often  as  the  causal  factor  in  175  aircraft  mishaps  due  to  human  error. 
Government,  university  and  industry  researchers  have  struggled  to  develop  answers  to  General 
McPeak’s  questions,  with  some  success.  Generic  definitions  of  this  elusive  construct  have  been 
developed,  and  specific  measurement  techniques  with  impressive  power  have  been  used.  An  off¬ 
line  situation  awareness  test  battery  incorporating  the  best  assessment  techniques  currently 
available  has  even  been  recommended. 

Yet,  the  general  consensus  continues  to  reflect  the  opinion  expressed  by  Shrestha,  Prince,  Baker, 
and  Salas  ( 1 994)  that  “...there  is  no  clear  operational  definition  of  situation  awareness  and  an 
understanding  of  situation  awareness  is  still  largely  incomplete.”  This  attitude  was  apparent  in 
papers  presented  at  the  “Second  Annual  Symposium  and  Exhibition  on  Situational  Awareness  in 
the  Tactical  Air  Environment”  at  the  Patuxant  River  Naval  Air  Station  on  June  3  and  4,  1997, 
and  in  a  Navy  Technical  Report  titled  Situational  Awareness  Guidelines.  In  both  of  these,  little 
attention  is  given  to  the  definition  or  measurement  of  SA.  Instead,  there  appears  to  be  an  implicit 
assumption  that  SA  (whatever  it  is)  would  be  enhanced  by  better  information  management  and 
presentation.  Thus,  the  focus  of  current  efforts  seems  to  have  shifted  toward  attempts  to  improve 
performance,  without  addressing  the  underlying  nature  of  the  SA  construct,  or  actually 
measuring  it.  In  brief,  at  this  time,  while  the  construct  of  situation  awareness  can  be  described  in 
conceptually  satisfying  terms,  there  is  no  well-defined  operational  definition  of  the  construct. 
Without  this,  it  is  difficult  to  see  how  individual  measures  of  SA  will  demonstrate  the  reliability 
and  validity  necessary  for  meaningful  applications. 

Beyond  the  question  of  an  operational  definition,  there  are  even  thornier  issues  involved  in  the 
development  of  a  broader  set  of  SA  metrics  than  have  been  used  in  the  past,  and  determining  how 
to  integrate  these  into  a  practical,  dynamic  simulation.  Current  measures,  while  showing 
reasonable  validity  (Fracker,  1991a),  have  been  criticized  because  some  are  limited  to  retrievable 
short-term  memory,  and  others  do  not  take  into  consideration  the  dynamic  nature  of  the  SA 
environment  (Sarter  and  Woods,  1991 ).  This  latter  point  is  especially  troubling  to  those 
attempting  to  develop  metrics  of  SA  useable  to  the  military  community.  If  laboratory'  studies 
yield  results  that  are  not  translatable  to  the  dynamic  operational  environment,  their  isolated 
“validity”  might  lead  to  erroneous  conclusions  and  catastrophic  system  designs. 

A  need  was  therefore  recognized  to  address  three  general  areas:  1)  the  development  of  a 
defensible,  theory-based  operational  definition  of  SA,  2)  the  development  of  an  expanded  set  of 
SA  metrics,  and  3)  the  development  of  a  dynamic  simulation  capability  into  which  the  SA 
metrics  can  be  embedded. 

The  present  report  describes  the  results  of  a  Small  Business  Innovation  Research  program 
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(Phases  I  and  II)  designed  to  address  these  goals.  A  brief  review  of  the  literature  on  situation 
awareness  has  been  presented  in  the  Phase  I  final  report  on  this  project  (O’Donnell,  Eddy, 
Cardenas,  Trafton,  and  Campbell,  1995),  and  will  not  be  repeated  here.  The  Phase  I  effort  also 
involved  development  of  a  proposed  cognitive-model-based  definition  of  SA,  and  this  is  repeated 
below,  since  it  forms  the  basis  of  a  series  of  proposed  SA  measures  that  were  implemented  and 
evaluated  in  Phase  11.  These  range  from  awareness  of  simple  spatial  location  and  environmental 
surroundings  to  the  existence  of  complex  cognitive  schemas.  The  final  design  and 
implementation  of  a  low-cost,  desk-top  flight  simulator  with  a  realistic  aerodynamic  flight  model 
was  then  carried  out,  and  this  development  is  described  in  detail  below.  This  simulation,  along 
with  its  embedded  SA  measures,  has  been  termed  the  Situation  Awareness  Flight  Training 
Evaluator  (SAFTE). 

Finally,  an  implementation  demonstration  designed  to  yield  initial  data  on  the  SAFTE  system 
was  carried  out,  and  is  documented  in  Section  Five.  Obviously,  for  any  complex  construct  such 
as  SA,  any  vehicle  purporting  to  assess  it  will  have  to  undergo  years  of  testing  and  modification 
before  rigid  scientific  criteria  can  be  met.  Questions  of  reliability,  validity,  stability,  and 
sensitivity  can  only  be  answered  in  any  final  sense  by  a  large  number  of  studies,  independently 
replicated.  However,  the  initial  demonstration  provides  at  least  a  preliminaiy  database  that  will 
allow  researchers  to  develop  appropriate  experimental  designs.  While  it  would  be  unrealistic  to 
believe  that  all  psychometric  questions  could  be  answered  within  this  effort,  the  demonstration 
described  in  the  appendix  to  this  report  should  provide  an  initial  foundation  on  which  further 
research  on  the  SAFTE  system  can  be  carried  out. 
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SECTION  TWO 


DEFINING  SITUATION  AWARENESS 


Operation  of  today’s  complex  aircraft  requires  almost  super-human  levels  of  performance 
sophistication  on  the  part  of  the  pilot  and  crew.  Over  the  years,  engineering  developments 
continually  made  manual  control  of  the  aircraft  easier  (although  the  degree  of  precision  in 
manual  control  has  not  decreased).  At  the  same  time,  however,  levels  of  cognitive  complexity 
required  to  carry  out  key  flight  operations  increased  dramatically.  In  effect,  what  began  as  a 
relatively  precise  manual  task  has  evolved  into  an  extremely  complex  cognitive  and  information 
processing  task,  while  not  significantly  reducing  the  requirements  for  control  precision. 
Essentially,  in  highly  complex  operations  (e.g.,  fighter  aircraft)  the  physical  demands  have  not 
been  reduced,  even  as  the  workload  and  information  processing  requirements  have  increased. 

It  is  not  surprising,  therefore,  that  the  questions  to  be  addressed  by  the  training  and  human 
factors  communities  have  similarly  become  more  complex.  Simple  manual  control  issues  have 
now  been  replaced  by  concerns  about  complex  constructs  that  have  proven  difficult  even  to 
define,  let  alone  to  measure.  Workload,  confidence  level,  and  decision  making  are  but  a  few  of 
these  complex  constructs.  They  have  the  essential  characteristic  of  being  multiply-determined 
entities  which  generate  or  “emerge”  from  a  complicated  interaction  among  many  individual 
factors. 

Among  such  emergent  constructs,  the  concept  of  “situation  awareness”  (referred  to  as  “SA”  in 
the  remainder  of  this  report)  has  perhaps  received  the  greatest  amount  of  recent  attention,  and  has 
proven  to  be  one  of  the  most  difficult  constructs  to  capture  empirically.  As  indicated  by  General 
McPeak,  most  individuals  feel  that  they  intuitively  understand  SA.  At  an  elementary  level,  it  is 
felt  to  be  a  consciousness  that  one  has  about  immediate  surroundings  and  immediate  tasks  at 
hand.  However,  even  the  most  superficial  reflection  reveals  that  this  notion  is  inadequate,  l.arge 
components  of  what  is  intuitively  considered  to  be  SA  are  simply  not  “conscious”  in  any 
traditionally  defined  way.  In  the  skilled  performer,  many  factors  which  permit  the  individual  to 
respond  to  the  present  environment  appropriately  (e.g.,  the  ability  to  access  mental  models  in 
long-term  memory,  or  the  ability  to  project  future  contingencies  based  on  present  and  past 
environments)  are  seldom  reflected  in  focal  consciousness. 

In  view  of  this,  any  attempt  to  define  SA  must  encompass  more  than  simple  focal  consciousness 
concepts.  In  the  past,  the  many  attempts  to  define  SA  have  had  a  great  deal  of  difficulty 
including  all  of  the  various  levels  and  factors  of  this  complex  construct,  at  least  in  ways  that  led 
to  objective  measures.  Representative  definitions  are  reviewed  below.  For  more  complete 
reviews,  beyond  the  scope  of  this  report,  see  Shrestha  et  al  (1994)  and  Dominguez  (1994). 

Representative  Definitions  of  Situation  Awareness 

One  way  of  classifying  definitions  of  SA  could  be  based  on  whether  the  focus  of  the  definition 
was  on  the  single  pilot  or  on  a  crew.  Taking  the  crew  interaction  approach,  Bolman  (1979) 
referred  to  SA  as  the  crew’s  theory  of  the  situation.  Bolman  placed  great  emphasis  on  the  ability 
of  each  crewmember  to  generate  “theories”  (or,  in  more  current  terminology,  mental  models), 
and  then  to  test  them  through  interaction  with  other  crewmembers,  while  confronting  and 
managing  conflicts.  Schwartz  (1990)  also  defined  SA  in  terms  of  the  perception  of  “factors  and 
conditions”  which  affect  the  aircraft  and  the  crew  over  a  defined  period. 
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Orasanu  (1990)  also  took  a  group  approach  to  understanding  SA,  embedding  the  construct  into  a 
decision-making  paradigm.  This  author  proposed  that  groups  must  utilize  certain  ‘‘supporting 
cognitive  components”  to  make  decisions.  These  included  situation  assessment,  metacognition, 
shared  mental  models,  and  resource  management.  The  component  of  situation  assessment  was 
defined  as  the  interpretation  and  appreciation  of  cues  important  to  the  recognition  of  a  problem 
requiring  decision  or  action.  The  actual  definition  of  SA,  then,  involved  the  crew’s  sensitivity  to 
these  cues.  With  this  orientation,  measures  of  SA  could  be  developed  based  on  the  types  of 
communication  among  crewmembers.  In  fact,  communications-based  measurement  of  SA  in  the 
multi-crew  environment,  along  with  rating  scales  and  peer  evaluation,  appear  to  be  the  currently 
popular  approaches  (e.g.,  Mosier  and  Chidester,  1991;  Brannick,  Prince,  and  Salas,  1992; 
Shrestha  a/,  1994). 

Definitions  of  SA  which  focus  exclusively  on  the  single-pilot  situation  tend  to  lean  more  toward 
the  individual’s  “knowledge”,  and  thus  tend  to  be  consistent  with  current  terminology  in 
cognitive  psychology.  For  instance,  Harwood,  Barnett,  and  Wickens  (1988)  defined  SA  as  the 
pilot’s  knowledge  of  a  dynamically  changing  situation.  Four  elements  were  seen  as  comprising 
that  knowledge:  1)  Where  —  location  of  objects  and  spatial  relations  among  them;  2)  What  - 
presence  of  objects  and  of  system  states;  3)  Who  —  the  command  structure;  and  4)  When  -- 
mission  events  along  a  time  continuum,  including  prediction  of  future  events. 

Using  the  cognitive  orientation  proposed  by  Rasmussen  (1986),  a  more  specialized  definition  of 
SA  has  been  proposed  by  Kass,  Herschler,  and  Companion  (1990).  They  defined  SA  as 
“...skilled  behavior  that  is  the  ability  to  extract,  integrate,  assess,  and  act  upon  task  relevant 
information.”  (Shrestha  et  al,  1994).  They  emphasized  the  point  that  highly  skilled  behavior  is 
based  on  the  acquisition  and  implementation  of  patterns  of  response,  rather  than  on  traditional 
stimulus-response  (S-R)  approaches. 

Perhaps  the  most  popular  current  definitions  of  SA  have  been  inspired  by  the  work  of  Endsley 
( 1 989)  and  Fracker  ( 1 988;  1 989).  Endsley  (1991)  defines  S  A  as  “the  perception  of  the  elemen  ts 
in  the  environment  within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning,  and 
the  projection  of  their  status  in  the  future”.  A  model  of  SA  developed  by  Endsley  (1991)  allows 
three  basic  levels  or  depths  of  SA:  1)  observation,  2)  integration,  and  3)  projection  of  events  into 
the  future.  The  first  level  --  observation  --  involves  a  recognized  sensory  input  that  is  stored 
through  iconic  and  echoic  memory.  The  conglomerate  of  these  observations  is  then  processed 
into  an  overall  picture  of  the  environment  in  the  integration  stage.  The  third  stage,  projection, 
involves  anticipation  of  the  outcomes  of  the  integration  carried  out  during  the  second  stage. 

Endsley  (1989)  also  enters  into  extensive  consideration  of  the  types  of  cognitive  processing  that 
are  carried  out  at  various  levels  of  S  A.  For  instance,  the  relevance  of  incoming  sensory 
information  may  be  compared  to  previous  experience,  and  may  trigger  more  focused  attention  on 
particular  aspects  of  the  environment.  Fracker  (1988)  emphasizes  this  focus.  Activities  are 
intimately  tied  to  working  memory  which,  in  turn,  may  be  related  to  long  term  memory  stores 
involving  more  complex  and  automatic  reactions  (schemas).  In  Endsley’s  view  (1989;  1991)  the 
actual  mechanisms  of  SA  are  intimately  tied  to  the  quality  and  quantity  of  schemas  available  to 
the  individual  in  long  term  memory.  If  appropriate  schemas  are  available  for  the  environmental 
and  task  demands,  short-term  memory  can  be  off-loaded,  and  the  individual’s  workload  is 
substantially  reduced.  If  appropriate  schemas  are  not  available,  processing  will  need  to  be 
carried  out  in  working  memory,  which  will  then  become  heavily  loaded. 

The  work  of  Endsley  and  Fracker  appears  to  have  influenced  a  popular  definition  of  SA  which  is 
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directly  relevant  to  the  flight  environment.  This  views  SA  as  a  pilots  continuous  perception  of 
self  and  aircraft  in  relation  to  the  dynamic  environment  of  flight,  threats,  and  mission  and  the 
ability  to  forecast,  then  execute  tasks  based  on  that  perception."  This  definition  has  been  widely 
reported  and  used  by  the  operational  community  in  the  Air  Force  (McMillan,  1994). 

One  of  the  more  comprehensive  recent  explorations  of  SA  has  been  provided  by  Vidulich, 
Dominguez,  Vogel,  and  McMillan  (1994).  They  distinguish  between  process  and  product  in  SA 
definitions.  Extraction  of  information  from  the  environment,  integration  of  information  into  an 
overall  mental  picture,  and  anticipation  of  future  events  are  emphasized  in  the  SA  definition 
(Dominguez,  1 994,  p.  1 1 ).  The  overall  approach  appears  to  divide  SA  into  four  cognitive 
activities:  1 )  extracting  information  from  the  environment;  2)  integrating  this  information  with 
relevant  internal  knowledge  to  create  a  mental  picture  of  the  current  situation;  3)  using  this 
picture  to  direct  further  exploration  in  a  continual  perceptual  cycle,  and  4)  using  this  information 
to  anticipate  future  events.  With  this  breakdown  of  the  SA  construct,  Vidulich  et  al  (1994)  have 
taken  a  major  step  toward  incorporating  several  aspects  of  existing  SA  definitions  into  one  that 
encompasses  both  the  process  and  product  of  SA. 

Toward  an  Operational  Definition  of  SA 

In  spite  of  some  obvious  differences  in  the  SA  definitions  reviewed  above,  there  are  some 
common  elements  (as  pointed  out  by  Dominguez,  1994)  which  might  serve  to  generate  a 
consensus.  Most  obviously,  current  concepts  of  SA  go  far  beyond  any  simple  notion  that  it 
consists  of  the  individual’s  conscious  “awareness”.  Most  current  definitions  have  included 
concepts  related  to  working  memory,  long  term  memory,  mental  models,  and  schemata.  Another 
common  thread  implied  by  most  current  definitions  is  that  SA  can  differentially  exist  on  several 
levels  or  depths  of  operation.  However  these  levels  are  defined,  there  appears  to  be  a  trend 
toward  understanding  that  S  A  may  involve  different  types  or  stages  of  information  processing. 

At  the  broadest  level  of  definition,  these  stages  are  divided  between  working  memoiy'  and  long 
term  memory.  The  general  concensus  is  that  if  the  more  working  memory  is  involved  in 
maintaining  SA,  the  workload  will  be  higher.  A  critical  implication  of  this  aspect  of  the 
definition  is  that  SA  will  have  to  be  measured  multi-dimensionally.  No  single  “holy  grail” 
assessment  technique  will  be  sufficient.  Finally,  the  work  of  Sarter  and  Woods  (1991),  as  well  as 
others,  suggest  that  SA  must  be  viewed  as  a  dynamic  process  that  must  be  assessed  as  an 
“embedded”  construct  in  the  context  of  actual  simulations. 

It  appears  from  the  above  discussion  that  any  definition  of  SA  broad  enough  to  encompass  the 
many  varieties  and  complexities  of  the  construct  would  have  a  great  deal  of  difficulty  being 
operationally  defined.  In  fact,  as  noted  above,  Shrestha  et  al  (1994)  have  concluded  that  even 
with  all  of  these  varied  definitions  of  SA,  no  operational  definition  currently  exists. 

“Operational”  in  this  context  refers  to  the  fact  that  theoretical  concepts  such  as  “knowledge  of 
...”,  “perception  of...”,  or  even  “awareness  of...”  are  used  to  define  the  construct,  without 
themselves  being  defined  in  measurable,  observable  terms.  While  such  approaches  certainly 
assist  in  understanding  the  nature  of  SA,  and  may  help  bound  the  overall  concept,  they  fail  to 
suggest  (let  alone  mandate)  specific  measurement  approaches  that  would  make  assessment  of  SA 
objective. 

The  goal  of  the  present  effort,  therefore,  was  first  to  develop  an  operational  definition  of  SA 
which  would  suggest  a  specific  set  of  measurement  approaches.  A  starting  point  for  such 
operational  definition  was  the  relative  agreement  among  scientists  that  SA  can  differentially  exist 
on  several  levels  or  stages  of  processing.  It  was  hypothesized  from  this  that  an  operational 
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definition  could  be  formulated  in  terms  of  different  types  or  levels  of  information  processing; 
carried  out  bv  the  operator. 

As  each  of  the  current  definitions  were  examined,  it  became  clear  that  they  address  levels  or 
“depth”  of  information  processing  components  such  as  sensory  registration,  perceptual 
organization  and  other  functions  within  working  memory,  long-term  memory,  and  even  highly 
overleamed,  automated  responses.  It  appears  reasonable,  therefore,  to  select  or  focus  on  such 
information  processing  stages  as  the  potential  operational  definition  of  SA.  Each  of  these  stages 
has  been  studied  empirically,  yielding  not  only  a  great  deal  of  information  about  the  stage  itself, 
but  also  about  possible  techniques  which  can  be  used  to  measure  the  individual  stage.  If, 
therefore,  SA  can  be  conceptualized  in  terms  of  the  level  or  depth  of  information  processing, 
there  is  a  significant  potential  that  it  could  be  measured  objectively. 

An  implication  of  this  approach  is  that,  because  SA  is  so  complex,  it  can  not  be  measured  b>  a 
simple,  uni-dimensional  "magic"  measure.  Instead,  the  construct  should  be  approached  from  the 
view  that  it  consists  of  multiple  levels  or  types  of  cognitive  processes,  which  combine  to  produce 
the  final  performance.  Measurement  of  the  SA  construct  will  therefore  involve  probes  of  many 
or  all  of  these  processes.  The  observable  “measure”  of  SA,  then,  will  be  the  degree  to  which  the 
person  can  demonstrate,  through  a  required  performance,  that  each  cognitive  process  is  capable 
of  being  utilized.  Ultimately,  these  measures  could  then  be  used  to  produce  a  cognitive  “profile” 
that  describes  that  individual’s  level  of  SA. 

Stated  most  succinctly,  it  is  proposed  that  an  appropriate  operational  definition  of  SA  is: 

The  types  or  levels  of  information  processing  performance  demonstrated  by  a 
particular  individual  under  a  given  set  of  environmental  and  task/goal 
demands. 

In  other  words,  it  is  suggested  that  metrics  of  SA  must  be  tied  to  the  capability  of  the  individual 
to  perform  well  within  a  given  task/goal  demand  situation,  where  these  tasks  or  goals  can  be 
objectively  related  to  the  level  of  information  processing  required  by  that  situation. 

The  above  definition,  while  somewhat  more  specific,  is  not  a  radical  departure  from  previous 
approaches.  It  is  consistent,  for  instance,  with  the  approach  of  Sarter  and  Woods  ( 1 991 ),  at  least 
in  the  sense  that  it  requires  a  dynamic  environment  be  used  to  test  the  total  concept  of  SA.  All 
measurement,  we  would  propose,  must  be  made  within  the  context  of  this  dynamic  environment. 

Indeed,  an  important  observation  by  Sarter  and  Woods  (1991)  pointed  the  way  to  making  this 
definition  truly  operational,  and  set  the  stage  for  subsequent  developments  in  the  present  effort. 
These  authors  point  out  that  in  traditional  cognitive  terms,  a  "mental  model"  does  not,  in  itself, 
constitute  SA.  Traditionally,  mental  models  refer  to  an  individual’s  representation  of  a  small 
piece  of  the  world  in  which  they  find  themselves.  For  example,  in  the  game  of  tic-tac-toe  a 
person  represents  the  game  with  two  pairs  of  crossed  lines,  Xs,  Os,  rules  of  play,  and  concepts  of 
winning  configurations. 

Situation  awareness,  on  the  other  hand,  is  viewed  as  an  open,  dynamic  system  that  involves  Ihe 
interplay  of  several  mental  models  (the  plane,  its  trajectory  through  space,  the  enemy’s  plane  and 
it’s  potential  flight  path,  the  weapon  systems  of  each,  etc.).  While  mental  models  might  be  a 
prerequisite  for  certain  types  of  SA,  their  existence  does  not  constitute  SA.  This  view  focuses 
attention  on  the  fact  that  the  actual  assessment  of  SA  must  be  tied  to  a  demonstration  that  the 
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individual  is  capable  of  utilizing  mental  models  or  schemas  in  a  given  situation.  In  other  words, 
it  is  not  sufficient  to  simply  know  what  to  do  in  a  given  situation;  if  the  pilot  lacks  the  abilits'  to 
perform  that  action,  then  no  SA  can  be  said  to  occur.  The  key  here  is  that  an  observable, 
measurable  behavior  should  be  required  from  the  operator  in  order  to  detemiine  that  a  certain 
level  or  type  of  SA  exists. 

This  recognition  led  naturally  to  investigation  of  existing  and  novel  assessment  techniques  that 
could  probe  various  levels  of  information  processing.  These  investigations  are  described  in  the 
next  section. 


SECTION  THREE 


■SITUATION  AWARENESS  MEASUREMENT  TECHNIQUES 
Existing  SA  Measurement  Techniques 

Although  many  techniques  have  been  used  to  probe  for  what  has  been  called  SA  ,  relatively  few 
have  been  used  for  more  than  one  or  two  situations,  or  have  been  tied  directly  to  a  theoretical 
foundation.  Fracker  (1991)  has  reviewed  SA  measures  extensively,  and  his  classification  forms 
the  foundation  of  the  discussion  presented  below.  Fracker  divided  SA  measures  into  three  broad 
categories:  subjective  measures,  explicit  measures,  and  implicit  measures.  Representative 
samples  of  each  of  these  types  are  described  below. 

Subjective  measures 

Subjective  measures  nominally  require  the  subject  to  rate  either  global  SA,  or  constructe 
believed  to  underlie  SA.  A  widely  used  subjective  scale  was  developed  by  Taylor  (1989),  and  is 
called  the  Situation  Awareness  Rating  Technique  (SART).  Three  factors  relating  to  SA  were 
identified  through  principal  component  analysis:  attentional  demand,  attentional  supply,  and 
situational  understanding.  Although  it  has  been  pointed  out  that  this  type  of  assessment 
confounds  workload  with  SA,  it  has  proven  valuable  in  some  situations.  Other  subjective  scales 
have  been  reported,  utilizing  information-theoretic  approaches  (Ambum,  1994)  or  other 
techniques. 

A  particularly  well-conceptualized  approach  to  measuring  SA  subjectively  has  been 
presented  by  Pew,  Adams,  and  Tenney  (1991).  These  authors  first  made  a  cogent  distinction 
between  the  process  of  SA  acquisition,  and  the  actual  awareness  resulting  from  that  process. 
Paralleling  this  concept,  they  suggest  two  different  approaches  to  measuring  SA.  One  is 
dedicated  to  isolating  the  process  itself,  and  involves  having  the  crew  “think  aloud”  during  or 
after  a  mission.  This  think  aloud  protocol  is  then  analyzed.  The  second  approach  involves 
examining  the  crew’s  performance  on  selected  missions  to  determine  retrospectively  what  the  .5.4, 
was.  In  the  specific,  the  authors  recommended  three  particular  measures.  The  first  one  involves 
computer  modeling  of  aircrew  performance.  This  model  is  then  compared  to  an  actual 
simulation  of  the  same  performance  in  order  to  isolate  the  knowledge,  procedures,  and  data  used 
by  the  crew  in  achieving  their  performance.  The  second  technique  proposed  by  Pew  et  o/.  (1991) 
involves  inserting  erroneous  or  anomalous  information  into  a  scenario  and  measuring  the  time 
that  it  takes  for  the  operator  to  recognize  the  unusual  information.  The  hypothesis  is  that  if  the 
operator  is  using  a  particular  mental  model,  this  detection  time  will  assess  the  quality  of  the 
model.  The  third  approach  involves  presenting  unexpected  situations  to  the  operator,  and 
assessing  the  time  that  it  takes  to  alter  the  original  operating  plan  or  mental  model  of  the  person. 
These  approaches,  though  promising,  do  not  appear  to  have  been  applied  in  actual  SA 
measurement  contexts,  although  they  have  been  used  in  other  contexts. 

Explicit  measures 

Explicit  measures,  as  discussed  by  Fracker  (1991),  can  be  divided  into  two  categories: 
retrospective  measures  and  memory  probes.  Retrospective  measures  involve  questioning  a 
subject  who  has  completed  a  mission  or  mission  segment  with  respect  to  relevant  items  that 
should  have  been  noted  during  the  performance.  They  represent,  therefore,  a  short-term  mernoiy' 
probe  with  some  time  delay  involved.  The  second  type  of  explicit  measure  has  gained 
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widespread  popularity  and,  in  so  far  as  anything  has  been  standardized  in  the  SA  literature,  has 
become  an  accepted  standard.  The  “probe  techniques”  described  by  Endsley  (1988)  involves 
exposing  the  subject  to  a  particular  scenario  and,  at  carefully  selected  points,  “blanking”  the 
scenario.  The  scenario  is  stopped,  or  salient  information  is  removed  from  the  screen.  The 
subject  is  then  questioned  concerning  the  information  in  order  to  determine  whether  he  or  she 
was  “aware”  of  it. 

Two  sub-categories  of  this  latter  approach  have  been  used.  In  one,  the  subject  is  directly 
questioned  about  the  occurrence  of  an  event.  In  the  other,  the  subject  is  asked  to  replace  or 
indicate  where  a  missing  object  should  be  placed.  Endsley  (1991)  suggests  that  the  former 
“questioning”  technique  may  be  appropriate  for  determining  the  “what”  question  -  that  is, 
whether  or  not  the  subject  recognized  that  an  event  had  occurred  or  that  a  particular  target  was 
present.  The  second  “replacement”  technique  is  particularly  appropriate  for  measuring  error  in 
the  subject’s  SA.  Since  it  is  possible  to  calculate  the  difference  between  the  actual  position  of 
the  probed  object  and  the  replacement  position,  interval-scaled  measures  of  error  can  be  obtained 
with  the  replacement  technique. 

While  the  two  probe  techniques  have  been  perhaps  the  most  widely  used  approaches  to 
assessment  of  SA,  there  have  been  some  disturbing  problems  reported.  Fracker  (1991)  reports 
that  some  probe  techniques  appear  to  be  reliable  while  others  are  not.  No  extensive  study  of 
validity  of  these  approaches  has  been  carried  out.  Fracker  points  out  that  probe  techniques  are 
still  after-the-fact  determinations.  As  such,  they  still  suffer  the  potential  problem  of  a 
discrepancy  between  what  actually  happened  during  the  mission  performance,  and  what  the 
subject  reconstructs  during  the  probe  itself. 

Implicit  measures 

Implicit  measures  of  SA  involve  projection  of  current  and  past  events  to  future 
conditions.  In  essence,  the  operator  is  required  to  respond  to  specific  questions  dealing  with  the 
occurrence  or  nonoccurrence  of  events  or  objects,  particularly  future  events  (Endsley,  1991)  or 
about  the  “envelope”  of  the  enemy’s  operation  (Fracker,  1991).  Fracker  has  also  pointed  out  that 
utilization  of  implicit  measures  necessitates  some  constraints  on  the  overall  variability  of  options 
in  the  real  situation.  As  such,  it  is  pointed  out  that  implicit  measures  of  SA  may  not  represent  an 
adequate  or  complete  range  of  conditions  for  a  given  situation. 

Integrated  SA  measures 

Implicit  and  explicit  measures  can  be  integrated  into  a  single  technique,  as  reported  by 
Endsley  (1987;  1988;  1990).  The  Situation  Awareness  Global  Assessment  Technique  (SAGAT) 
blanks  the  simulation  at  some  random  point,  and  presents  a  series  of  questions  intended  to  reflect 
the  operator’s  SA  requirements  and  knowledge.  These  questions  are  randomly  taken  from  a 
larger  set  of  questions,  and  a  composite  SAGAT  score  is  then  developed  by  comparing  the 
subject’s  answers  to  the  real  situation.  Endsley  maintains  that  the  SAGAT  provides  a  highly  face 
valid  global  measure  of  SA  which  is  objective,  and  which  reflects  the  instantaneous  condition  of 
the  operator. 

However,  Sarter  and  Woods  (1991)  criticized  the  SAGAT  technique  as  being  intrusive, 
and  as  prompting  specific,  static  information  which  will  differ  in  essential  ways  from  that  which 
would  actually  be  experienced  in  the  dynamic  flight  situation.  These  authors  believe  that 
memory  probe  measures  in  general  tap  a  limited  aspect  of  the  overall  SA  construct.  In  fact,  they 


13 


maintain  that  any  post-hoc  data  collection  will  fail  to  tap  (and  may  even  distort)  unconscious 
processes  that  were  actually  present  during  the  task  performance.  Sarter  and  Woods  (1991) 
suggest  that  SA  can  only  be  assessed  in  the  context  of  a  complex,  dynamic  simulation  which  has 
been  specifically  developed  to  probe  well-defined  SA  issues.  In  other  words,  scenarios  must  be 
developed  which  span  a  range  of  situations,  from  those  with  low  level  requirements,  to  those 
presenting  complex  interacting  situations.  They  suggest  that  “...by  embedding  specific  tasks  that 
tap  knowledge  of  the  different  dimensions  of  situation  awareness,  it  is  possible  to  determine  what 
specific  information  the  pilot  must  be  aware  of  at  a  particular  time.” 

SA  Measures  Incorporated  Into  the  Situation  Awareness  Flight  Trainer  and  Evaluation  (SAFTE) 
System 


A  meeting  was  convened  at  Brooks  AFB  in  May  1995,  to  consider  the  SA  measures  that 
had  been  recommended  during  Phase  I  of  this  effort  (O’Donnell,  Eddy,  et  al,  1995).  NTI 
scientists  and  programmers,  along  with  pilot  consultants  and  Government  representatives, 
attended  this  meeting.  Each  candidate  SA  metric  was  discussed  from  several  viewpoints.  The 
rationale  for  including  it  in  the  original  recommendations  was  presented  and  evaluated.  If  the 
rationale  was  considered  defensible,  practical  issues  of  how  the  measure  could  be  obtained  v>'ere 
then  suggested  and  evaluated.  Programming  difficulty  was  considered  as  part  of  this  evaluation. 
Finally,  a  consensus  decision  on  whether  the  measure  should  actually  be  included  in  the  S.AFTE 
system  was  made. 

In  line  with  the  SA  definition  proposed  above,  the  general  orientation  in  selecting  SA 
metrics  was  to  include  a  range  of  techniques  that  probed  as  many  information  processing 
components  as  possible.  Further,  the  original  concept  of  SAFTE  was  to  develop  a  set  of  metrics 
that  could  be  “embedded”  into  the  flight  simulations.  These  metrics  were  conceptualized  as 
relatively  discrete  measures,  each  of  which  would  contribute  a  different  kind  of  information 
about  the  individual’s  SA  at  the  time  of  measurement.  They  would  then  provide  a  multi¬ 
dimensional  description  of  the  person’s  state  of  SA  (which,  after  all,  is  a  multiply  determined 
construct).  This  description  would  be  based  on  which  capabilities  he  or  she  can  demonstrate, 
and  the  degree  to  which  each  one  is  demonstrated.  In  other  words,  the  intent  was  to  provide  a 
measurement  tool  that  is  able  to  define  the  profile  of  an  individual’s  SA. 

In  practice,  SA  assessment  within  the  SAFTE  system  consists  of  a  set  of  flight  scenarios, 
during  which  multiple  SA  probes  are  employed.  The  combination  of  probe  performances  on 
which  the  subject  performed  adequately  defines  his  or  her  level  of  SA  for  that  type  of 
environment  and  situation.  The  actual  considerations  and  measures  incorporated  into  the  SAFTE 
system  are  described  below. 

SA  MEASURE  #1.  ENVIRONMENTAL  STATUS  AWARENESS 


Rationale  for  the  Measure 

Virtually  all  information-processing  theories  agree  that  appropriate  registration  of 
sensory  conditions  provides  the  basic  foundation  for  later  processing.  In  the  case  of  SA  metrics, 
this  must  be  taken  to  include  not  only  basic  sensory  registration,  but  some  level  of  conscious 
awareness  of  the  sensory  environment.  Stated  most  simplistically,  this  level  of  SA  processing 
says  that  the  person  must  be  taking  in  the  relevant  aspects  of  the  environment  and,  at  some  level 
must  be  “aware”  of  that  environment.  This  is  similar  to  what  Endsley  (1990)  describes  as  “level 
1”  situation  awareness. 
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Operationally,  this  type  of  awareness  is  critical  in  a  variety  of  situations.  Most  notably, 
in  an  air-to-air  combat  situation,  the  pilot  must  often  engage  one  bandit  with  “focal”  attention, 
while  still  processing  the  general  position,  number,  and  movement  of  others.  When  the  first 
engagement  is  completed,  the  pilot  must  be  able  to  know  the  approximate  location  of  the  next 
bandit  to  be  engaged.  This  is  the  type  of  SA  that  will  be  probed  by  the  first  SA  metric. 

In  the  present  case,  it  is  proposed  that  this  type  of  SA  incorporates  processing  beyond  the 
iconic  and  echoic  stages  in  that  some  degree  of  “awareness”  must  be  present.  It  should  be  noted 
that  this  awareness  need  not  be  in  the  forefront  of  consciousness.  Rather,  the  criterion 
established  for  this  type  of  awareness  is  simply  that  the  information  is  retrievable.  In  other 
words,  either  simultaneously  with  its  occurrence  or  subsequently,  the  person  can  remember  and 
report  the  information  concerning  the  immediate  environment. 

Approach  to  Measuring  Environmental  Status  Awareness 

In  terms  of  this  type  of  awareness,  it  is  hypothesized  that  the  person  should  be  able  to 
report  on  the  existence  of  significant  aspects  of  the  environment.  The  goal  here  is  simply  to 
determine  whether  the  person  is  able  to  retrieve  sensory  input  information  from  short-term 
memory.  No  attempt  is  made  to  elicit  precise  location  information  or  complex  interpretation  of 
the  data,  and  no  projection  into  the  future  is  required.  Thus,  this  level  of  probe  is  intended 
simply  to  measure  the  person’s  range  of  “awareness”  of  the  environmental  conditions.  Such 
“awareness”  is  typically  probed  either  with  retrospective  or  concurrent  “explicit”  measures  of  SA 
(Fracker,  1991).  Because  there  is  some  controversy  concerning  whether  retrospective  (after-the- 
mission)  measures  confound  momentary  and  reflective  SA,  the  concurrent  measurement 
approach  was  incorporated  into  the  SAFTE  system,  as  described  below. 

As  a  further  refinement  of  this  type  of  measure,  Eubanks  and  Killeen  (1983)  and  .Amburn 
(1994)  have  suggested  that  the  probe  be  structured  with  “yes-no”  responses  in  order  to  perm  it 
information-theoretic  analyses.  In  effect,  since  each  response  reflects  a  “positive  hit  (a  correct 
yes  answer),  a  “negative  hit”  (a  correct  no  answer),  a  “positive  miss”  (an  incorrect  no  answer),  or 
“false  alarm”  (an  incorrect  yes  answer),  the  data  lend  themselves  to  a  2-by-2  matrix.  From  this, 
following  Kantowitz  &  Sorkin  (1983),  the  sensitivity  of  the  subject  (d’)  can  be  calculated  for  the 
given  experimental  condition.  In  other  words,  standard  information  theoretic  measures  can  be 
calculated.  If  desired,  even  entire  receiver-operating-characteristic  (ROC)  curves  could  be 
determined  for  a  series  of  experimental  conditions  to  account  for  other  sources  of  variance  than 
SA.  The  “sensitivity”  of  the  operator  reflects  his  or  her  ability  to  detect  an  environmental 
condition.  Fracker  (1991)  recommends  this  approach  as  “...an  empirical  and  an  intuitively 
reasonable  measure  of  awareness  for  a  particular  kind  of  event”.  The  SAFTE  system  therefore 
collects  all  data  in  this  measure  (and  others  described  below)  in  the  form  of  questions  that  can 
either  be  answered  “yes”  or  “no”.  The  individual  user  can  then  decide  whether  to  simply 
calculate  percent  correct,  or  to  apply  the  information  theoretic  approach. 

SAFTE’s  Technique  for  Measuring  Environmental  Status  Awareness 

The  technique  used  in  SAFTE  for  this  and  some  later  SA  measures  is  one  generally 
referred  to  as  the  “freeze”,  “memory  probe”,  or  “blanking”  approach  (Fracker,  1991;  Endsley, 

1 994).  In  this,  the  screen  of  the  simulator  goes  blank,  either  at  defined  mission  times  or  specific 
mission  events.  A  short  question  appears  on  the  screen  asking  for  specific  information  about 
whether  certain  conditions  were  or  were  not  present  at  the  time  of  the  blanking.  The  information 
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requested  simply  involves  whether  a  given  situation,  event,  or  object  was  present  or  not. 

Although  general  information  about  the  location  of  an  object  might  be  requested,  no  precise 
localization  is  requested  (as  will  be  done  in  later  measures).  Examples  of  questions  that  were 
used  in  the  demonstration  project  (see  Section  Five)  are  provided  below,  and  the  full  list  of 
questions  used  is  given  in  the  Appendix  7. 

-  At  900  MSL,  ask:  “YOU  ARE  NOW  ABOVE  800  FEET”. 

-  At  9  min.  into  the  mission,  ask:  “YOU  ARE  LESS  THAN  8  MINUTES  INTO 
THIS  MISSION.” 

-  At  5  NM  from  the  target,  ask:  “YOU  SHOULD  SELECT  AB  NOW!” 

-  PRIOR  TO  WP  #1,  ask:  “YOU  SHOULD  HAVE  ALREADY  SELECTED 

AIR-TO-AIR  MODE.” 

-  During  an  early  portion  of  the  SECOND  “outbound”  leg  of  a  Combat  Air  Patrol 

(CAP)  mission,  ask:  “YOU  ARE  NOW  ON  AN  OUTBOUND  LEG  OF  THE 

CAP  PATTERN.” 

-  Within  5  seconds  after  the  first  launch  of  a  MRM  on  a  bomber,  ask:  “THERE 
IS  NO  SAM  SITE  TARGETING  YOU  AT  THIS  TIME.” 

In  any  particular  use,  the  questions  to  be  asked  will  depend  totally  on  the  scenarios  being 
used  by  the  experimenter  or  trainer.  Therefore  the  SAFTE  software  provides  a  simple  way  to 
generate  and  insert  new  questions  into  either  the  “canned”  scenarios  or  into  those  generated  by 
the  user.  However,  the  sets  of  questions  used  in  the  demonstration  project  can  also  be  used  as 
they  exist  in  the  delivered  SAFTE  product.  In  other  words,  the  system  is  robust  enough  to  permit 
researchers  a  great  deal  of  freedom  in  such  choices. 

SA  MEASURE  #2.  AWARENESS  OF  FUTURE  ACTIONS 

Rationale  for  the  Measure 

The  next  type  of  SA  measure  also  uses  the  blanking  technique  proposed  by  Endsley 
(1994),  and  described  under  SA  MEASURE  #1  above.  In  this  case,  the  questions  asked  of  the 
operator  during  the  blanked  period  do  not  simply  refer  to  the  presence  or  absence  of  an 
environmental  condition,  but  rather  request  information  about  an  activity  to  be  carried  out  in  the 
future.  In  other  words,  the  operator  is  not  simply  queried  about  whether  an  object  or  condition 
exists,  but  is  asked  to  anticipate  an  event  or  condition  that  will  happen  later. 

Although  this  and  the  previously  described  measure  appear  to  share  a  great  deal  in 
common,  it  is  argued  that  they  actually  measure  different  levels  of  SA.  At  very  least,  responding 
correctly  to  questions  involving  future  action  requires  an  ability  to  “project”  events  based  on 
current  conditions.  Thus,  it  would  appear  that  this  type  of  awareness  requires  considerably  more 
working  memory  capacity  than  simply  reporting  whether  the  object  or  condition  was  present.  In 
practical  terms,  this  type  of  probe  should  be  operationally  relevant  in  those  situations  where  the 
individual  might  become  pre-occupied  with  the  current  task  to  the  extent  that  future  actions  are 
jeopardized. 
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Approach  to  Measuring  Awareness  of  Future  Actions 


The  actual  measurement  of  this  level  of  SA  is  virtually  identical  to  that  described  for  SA 
Measure  #1 .  The  only  difference  is  that  questions  probe  what  the  pilot  has  been  instructed  to  do 
in  the  future.  In  selecting  such  questions,  a  major  consideration  should  be  the  activity  that  is 
going  on  when  the  question  is  asked.  At  the  simplest  level,  the  question  could  ask  when  the  next 
turn  should  be  initiated  while  the  aircraft  is  in  straight  and  level  flight  (low  activity).  A  more 
challenging  type  of  question  would  involve  asking  about  a  turn  that  might  be  required  while  the 
aircraft  is  ascending  or  descending  (moderate  activity).  An  extreme  challenge  would  be  to  ask  a 
question  while  the  pilot  was  in  the  middle  of  a  pop-up  bomb  delivery  (high  activity). 

SAFTE’s  Technique  for  Measuring  Awareness  of  Future  Actions 

The  SAFTE  system  measures  awareness  of  future  actions  with  examples  of  each  of  the 
above  types  of  questions.  In  all  of  the  procedures,  the  screen  is  blanked,  and  a  question  is  posed 
on  the  screen.  The  question  requires  the  operator  to  give  “yes-no”  answers.  As  described  above, 
these  answers  can  be  analyzed  as  percent  coirect,  or  with  information  theoretic  techniques. 
Examples  used  in  the  demonstration  study  are  given  below,  with  the  complete  list  presented  in 
Appendix  7. 

-  At  15,000  MSL,  ask:  “YOU  ARE  TO  LEVEL  OFF  AT  20,000  FT.” 

-  Immediately  after  WP2  has  been  entered,  ask:  “YOUR  NEXT  ALTITUDE  CHANGED 

WILL  BE  TO  GO  TO  600  MSL.” 

-  At  9:45  min.  into  the  mission,  ask:  “AFTER  YOU  REACH  WP3,  YOU  SHOULD 

DESCEND  TO  540  MSL.” 

-  Immediately  after  subject  begins  the  pull-down  maneuver  in  a  pop-up  bomb  attack,  ask: 

“YOUR  FINAL  RELEASE  ALTITUDE  SHOULD  BE  10,000  MSL.” 

-  If  altitude  goes  below  7,800  MSL,  ask:  “YOU  SHOULD  PICKLE  AS  SOON  AS  THE 

PIPPER  REACHES  THE  TARGET.” 

-  At  5  NM  prior  to  WP  5,  ask:  “YOUR  NEXT  TASK  WILL  BE  TO  TURN  TO  WP  5 

AND  MAINTAIN  ALTITUDE.” 

SA  MEASURE  #  3.  AWARENESS  OF  INTEGRATED/PROJECTED  INFORMATION 

Rationale  for  the  Measure 


This  level  of  SA  encompasses  the  subject’s  ability  to  report  considerably  more  dynamic 
aspects  of  the  stimulus  environment  than  simple  existence  of  objects,  conditions  or  programmed 
future  action.  Specifically,  the  probes  in  this  measure  are  designed  to  determine  whether  the 
person  is  capable  of  reporting  information  that  is  “derivative”  in  nature.  In  this  context,  the  term 
“derivative”  is  defined  as  requiring  the  integration  of  a  number  of  sensory/perceptual  elements 
into  a  hypothesis  about  a  present  or  future  state  of  the  environment.  In  other  words,  this  level  of 
SA  results  in  “information”  (or  a  belief)  that  a  state  exists,  or  is  about  to  occur,  in  the 
environment  which  cannot  be  confirmed  by  the  information  available  in  the  environment  at  this 
moment  in  time.  It  is  an  extrapolation  or  inference  based  not  only  on  the  data  immediately 
available,  but  also  on  what  has  gone  on  previously,  and  the  operator’s  expectations  about  what 
will  go  on  in  the  future. 
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Approach  to  Measuring  Integrated/Proiected  Information  Awareness 

The  general  concept  of  this  level  of  SA  is  based  on  the  assumption  that  the  pilot,  at 
various  levels  of  SA,  progresses  from  a  simple  awareness  of  “what  is”  or  “what  will  be”,  to 
“what  might  be”.  In  the  previous  assessment  techniques,  probes  have  determined  whether  the 
person  knows  that  certain  environmental  objects  exist,  and  what  he  or  she  is  supposed  to  do  in 
the  future.  In  the  present  level  of  SA,  the  probe  is  directed  to  determining  whether  the  pilot 
knows  what  future  conditions  will  exist  given  what  is  happening  now.  In  other  words,  given 
certain  assumptions,  what  will  my  present  state  cause  to  happen  in  the  future?  At  this  stage, 
interest  is  in  whether  the  pilot  can  predict  these  future  events  at  a  relatively  simplistic  level. 

From  a  conceptual  viewpoint,  this  level  of  SA  requires  the  pilot  not  only  to  “know” 
where  relevant  elements  of  the  environment  are  (recognizing  that  these  may  not  be  in  the 
forefront  of  consciousness),  but  also  to  integrate  immediate  past  experience  (the  angular 
movements  of  own  and  enemy  aircraft)  and  present  position,  and  then  to  project  these  into  a 
future  state.  It  should  be  noted  that  this  level  of  SA  requires  somewhat  more  sophisticated 
“training”  of  the  person,  over  and  above  basic  flying  skills,  in  the  sense  that  it  requires  that  the 
person  have  extensive  experience  with  the  specific  predictions  unique  to  the  flight  environment. 

SAFTE's  Techniques  for  Measuring  Integrated/Proiected  Information  Awareness 

Two  types  of  questions  were  used  under  this  general  level  of  SA:  specific  event 
projections,  and  more  complex  “derivative”  event  projections.  The  first  type  is  similar  to  the 
“projection”  probes  described  by  Endsley  (1991b).  These  are  blanking  techniques  that  are 
administered  in  the  same  way  as  the  other  blanking  techniques  incorporated  in  SA  Measures  I 
and  2  above.  In  this  case,  however,  the  probe  questions  deal  with  projections  of  specific  events 
into  the  future.  This  presumably  goes  beyond  any  of  the  types  of  probe  described  above.  The 
goal  here  is  to  identify  a  class  of  complex  predictive  constructs  that  experienced  pilots  use  in 
deciding  how  to  operate  the  aircraft.  It  is  still  relatively  primitive  in  the  sense  that  it  involves 
working  memory  to  a  very  large  (not  necessarily  exclusive)  extent.  In  effect,  this  class  consists 
of  complex  inferences  by  the  pilot,  based  on  the  immediate  past  and  the  pilot  s  expectations 
about  his  or  her  own  behavior,  and  which  could  be  verbally  reported  during  or  shortly  after  thg 
event. 


The  second  category  of  question  under  this  general  technique  involves  a  more  complex 
derivative  awareness  by  the  pilot.  Interviews  with  the  pilot  consultants  revealed  that  there  aie 
certain  cognitive  “bases  of  flight  operations”  which  are  critical  to  combat  success.  A  prime 
example  of  such  a  basis  can  be  termed  “perception  of  the  enemy’s  plane  of  operation.”  In  air-to- 
air  combat,  it  is  critical  to  maintain  awareness  of  the  plane  described  by  the  enemy  aircraft’s 
current  linear  momentum  (Shaw,  1985).  In  effect,  in  a  dynamic  engagement  between  aircraft,  the 
“horizontal”  plane,  which  is  normally  represented  by  the  earth’s  horizon,  changes  every  time  an 
enemy  aircraft  makes  a  maneuver  (and  the  pursuit  aircraft  responds  to  that  maneuver).  At  one 
moment,  the  enemy  may  be  flying  or  turning  in  a  plane  that  is  completely  parallel  to  the  horizon. 
In  the  next  instant,  the  enemy  may  enter  a  diving  turn  in  which  the  plane  of  operation  now 
changes  to  a  45-degree  angle  relative  to  the  horizon.  It  is  incumbent  on  the  pilot  to  be  aware  of 
the  enemy’s  plane  of  operation  at  all  times  in  order  to  carry  out  a  responsive  maneuver. 

In  terms  of  cognitive  theory,  these  types  of  “awareness”  involves  a  complex  synthes  is  of 
several  individual  kinds  of  present  and  past  information  which  are  integrated  at  a  very  high  (late) 
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level  of  working  memory.  For  example,  in  determining  a  “plane  of  operation”,  the  pilot  must 
perceive  a  motion  vector  of  the  enemy  aircraft  for  some  period  of  time,  consider  the  present 
tactical  situation,  and  project  this  into  a  probable  plane  of  motion  of  the  enemy  in  the  immediate 
future.  The  integration  of  these  separate  activities  must  be  carried  out  (and  appear  to  the 
experienced  pilot)  as  a  single  percept.  For  this  reason,  probe  techniques  that  assess  the 
individual’s  ability  and  current  capability  to  maintain  this  type  of  awareness  represent  a  different 
level  of  SA  than  those  discussed  previously. 

Examples  of  these  types  of  question  used  in  the  demonstration  study  include  the 
following: 

-  At  9:15  minutes  into  the  mission,  ask:  “YOU  SHOULD  HAVE  LESS  THAN  1 

MINUTE  LEFT  BEFORE  ARRIVING  AT  WP3.” 

-  While  the  pilot  is  following  the  flight  director,  ask:  “THE  FLIGHT  DIRECTOR 

APPEARS  TO  BE  GIVING  FALSE  INFORMATION.” 

-  During  ascent  (at  any  point)  ask:  “ROLLING  LEFT  IS  THE  QUICKEST  WAY  TO 

BRING  THE  TARGET  VERTICAL  RELATIVE  TO  THE  HUD.” 

-  When  aircraft  is  3  min.  from  WP3,  ask:  “YOU  HAVE  TRAVELED  MORE  TH.\N 

HALF  THE  DISTANCE  BETWEEN  WP2  AND  WP3.” 

-  When  aircraft  has  turned  15  deg.,  ask:  “YOU  ARE  AT  LEAST  HALFWAY 

THROUGH  YOUR  REQUIRED  TURN.” 

Other  general  examples  of  questions  that  could  be  generated  in  these  categories  are  fuel 
management,  armament  selection,  and  projections  of  future  threat  configurations.  Probe 
questions  (or  even  measured  flight  responses)  which  tap  these  various  kinds  of  information 
obviously  will  be  appropriate  for  different  kinds  and  portions  of  missions.  A  representative 
sampling  of  these  data  should  provide  a  comprehensive  indication  of  the  pilot’s  ability  to 
maintain  this  level  of  SA. 

SA  MEASURE  #4.  SPATIAL  ORIENTATION/DISORIENTATION 

Rationale  for  the  Measure 


Gillingham  (1992)  and  many  operational  pilots  have  stated  that  one  of  the  basic  causes 
of  a  loss  of  SA  is  spatial  disorientation  (SD).  In  a  most  fundamental  sense,  the  foundation  for  SA 
in  the  aircraft  is  that  the  pilot  knows  how  the  craft  is  positioned  and  moving  with  respect  to  the 
earth.  If  this  basic  awareness  does  not  exist  (at  least  in  normal  flight  situations)  it  is  unlikelj'  that 
more  complex  situations  will  be  clearly  apprehended.  Spatial  disorientation  is,  in  fact,  defined  as 
the  “erroneous  sense  of  one’s  position  and  motion  relative  to  the  plane  of  the  earth’s  surface” 
(Gillingham,  1992).  This,  of  course,  is  directly  related  to  the  above  discussion  of  derivative 
event  projection,  except  that  it  refers  to  perception  of  one’s  self,  rather  than  of  an  outside  entiw. 

It  appears,  then,  that  one  of  the  most  basic  measures  of  the  pilot’s  SA  should  be  to  determine 
whether  the  aircraft’s  position  and  motion  are  correctly  perceived. 

Approach  to  Measuring  Spatial  Orientation/Disorientation 

In  flight,  orientation  is  maintained  by  monitoring  information  from  the  flight 
instruments.  Angular  positions  are  bank,  pitch,  and  heading.  The  corresponding  angular 
velocities  are  the  rates  of  roll,  pitch,  and  turn  (yaw).  Altitude  is  the  linear  position  parameter; 
and  airspeed,  slip/skid  rate,  and  vertical  velocity  are  the  linear  velocity  parameters.  Given  the 
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above,  it  can  be  concluded  simply  that  spatial  orientation  is  nothing  more  than  the  correct 
awareness  of  the  control  and  performance  flight  parameters.  Any  measure  of  spatial  orientation 
then  becomes  a  function  based  on  the  ability  to  determine  how  well  the  subject  maintains  desired 
performance.  For  instance,  if  the  pilot  is  required  to  accomplish  a  task  which  demands 
maintenance  of  airspeed,  and  all  other  distracters  are  removed,  then  the  pilot  can  be  said  to  he 
spatially  disoriented  when  airspeed  consistently  deviates  outside  of  an  accepted  tolerance  for 
normal  performance  (e.g.,  +/-  10  knots).  Maintenance  of  precise  airspeed  by  a  trained  pilot  is  a 
direct  reflection  of  awareness  in  this  case. 

SAFTE’s  Techniques  for  Measuring  Spatial  Orientation/Disorientation 

Since  there  are  no  accepted  standard  ways  to  measure  SD  as  an  index  of  SA,  methods 
which  have  been  developed  to  assess  the  effects  of  variations  in  flight  symbology  on  flight 
performance  were  adapted  to  the  present  use.  These  techniques  have  been  described  in  detail  by 
Weinstein  and  Ercoline  (1991),  and  have  been  used  in  high  fidelity  simulators  and  in  actual 
flight.  Specifically,  SD  is  measured  in  two  ways. 

A.  Precision  instrument  control  task. 

In  the  first  technique,  termed  the  ‘‘precision  instrument  control  task”,  aircraft 
performance  is  measured  against  pre-briefed  performance  characteristics  (airspeed, 
altitude,  etc.)  in  those  situations  where  these  can  be  defined.  This  is  accomplished 
by  determining  the  root-mean-square  (RMS)  error  of  those  flight  parameters  required 
for  a  given  mission.  The  subject  is  briefed  that  minimum  deviations  about  a  “target” 
flight  parameter  are  the  desired  outcome.  He/She  must  then  apply  nonual  instrument 
crosscheck  procedures  to  integrate  all  the  needed  information  to  minimize 
deviations. 

Traditionally,  this  method  has  been  used  where  one  flight  parameter  is  of  interest  or 
when  each  flight  parameter  is  treated  independently  of  the  others.  Only  recently 
have  attempts  been  made  to  determine  the  effect  one  parameter  has  on  another. 

When  dealing  with  a  complex  construct  such  as  SA,  of  course,  these  interactive 
effects  will  be  of  great  interest  to  the  experimenter.  The  capability,  within  the 
SAFTE  system,  to  record  many  flight  parameters  and  store  them  in  a  readily 
accessible  way  (see  below),  will  facilitate  many  types  of  investigations  using  this  SA 
Metric. 

B.  Unusual  attitude  recovery. 


In  this  procedure,  the  pilot  is  taught  a  procedure  prior  to  testing  which  is  unique  to 
one  of  four  classes  of  spatial  orientation: 

1  UA  Number  1 .  If  presented  with  a  nose  high  situation,  the  pilot  must  confirm 
that  the  unusual  attitude  exists,  then  initiate  a  stick  and  throttle  input  dependent 
upon  the  bank.  If  the  display  depicts  an  inverted  bank  (>90  degrees),  then  the 
pilot  is  to  adjust  power  to  maintain  airspeed,  roll  to  a  completely  inverted  bank 
(avoiding  negative  G-forces),  and  pull  the  nose  toward  the  horizon.  Once 
oriented  close  to  a  wings  level,  inverted  attitude,  the  pilot  is  to  roll  the  aircraft 
upright  (e.g.,  within  +/-  8  degrees  of  pitch  and  +/-  5  degrees  of  bank). 
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2)  UA  Number  2.  Given  the  same  nose  high  situation,  but  with  the  bank  less  than 
90  degrees,  the  pilot  is  to  either  maintain  the  bank  (allowing  the  nose  to  drop 
toward  the  horizon)  or  increase  bank  toward  an  inverted  position,  while 
maintaining  positive  G-forces.  Recoveiy  is  complete  when  the  aircraft  is  upright 
and  in  the  same  orientation  as  described  in  1 )  above. 

3)  UA  Number  3.  If  the  pilot  sees  an  orientation  that  is  nose  down  and  less  than  90 
degrees  of  bank,  the  correct  recovery  procedure  is  to  roll  in  the  shortest  direction 
toward  an  upright  condition  while  pulling  the  nose  up  and  adjusting  the  power  as 
necessary.  Recovery  is  defined  as  above. 

4)  UA  Number  4.  If  there  is  a  nose  down  condition,  and  the  bank  is  greater  than  90 
degrees,  the  pilot  must  not  pull  the  nose  up  until  the  bank  is  less  than  90  degrees. 
Again,  recovery  is  defined  as  above. 

The  data  collection  procedures  for  these  two  techniques  are  reasonably  straightforward 
within  SAFTE.  The  precision  instrument  control  task  involves  simply  defining  the  flight 
parameters  of  interest  for  a  given  mission  scenario,  and  then  briefing  the  pilot  to  maintain  that  set 
of  parameters.  This  can  be  made  as  complexes  as  desired  by  introducing  distracters  during  the 
mission,  in  order  to  determine  whether  the  pilot  can  maintain  spatial  orientation.  The  researcher 
will  define  analyses  (either  the  standard  RMS  error  provided  by  SAFTE  or  other  statistics). 

For  the  unusual  attitude  recovery  task  in  SAFTE,  the  aircraft  is  presented  in  one  of  the 
above-defined  unstable  orientations  after  a  screen  blanking  or  freeze  (as  described  above  for  SA 
MEASURES  1,  2,  or  3).  This  simulates  an  event  that  could  have  occurred,  for  example,  during  a 
“blackout”  or  short  period  of  loss-of-consciousness  (LOG).  In  other  words,  the  subject  will  have 
just  been  probed  about  some  external  condition  while  the  screen  was  “blanked”.  After 
responding,  the  simulation  re-appears.  However,  the  aircraft  may  be  in  one  of  the  unusual 
attitudes  described  above.  Dependent  measures  include  time  to  initial  stick  input,  accuracy  of 
stick  input  (per  AFM  51-37),  and  total  time  to  recovery. 

SA  MEASURE  #5.  MINIMUM  INFORMATION  REQUIREMENT 

Rationale  for  the  Measure 


Most  definitions  of  SA  imply  that  specific  kinds  of  information  are  being  processed 
(while  others  are  not)  at  any  point.  The  thrust  of  the  entire  present  approach,  in  fact,  is  to 
operationally  define  SA  in  terms  of  this  distinction.  It  is  proposed  that  a  person’s  SA  can  be 
described  by  determining  what  information  the  pilot  is  capable  of  processing.  While  it  is 
desirable  to  have  metrics  that  probe  specific  types  of  information  processing  defined  by  the 
investigator,  it  is  also  desirable  to  include  measures  which  allow  the  subject  to  reveal  what  types 
of  information  he  or  she  requires  to  carry  out  the  task.  In  other  words,  instead  of  asking  pre¬ 
defined  questions,  these  metrics  should  reveal  whatever  information  requirements  exist  for  an 
individual  at  a  given  time. 

Approaches  to  Measuring  Information  Requirements/Processing 

The  first  four  measurement  categories  proposed  above  attempt  to  arrive  at  decisions 
about  specific  types  of  information  processing  by  relatively  indirect  means.  The  present 
measurement  category  approaches  the  same  task  with  a  more  global  and  direct  set  of  methods. 
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The  subject  is  asked  directly  to  indicate,  through  behavior,  what  information  is  required  at  any 
point.  From  these  behavioral  responses,  it  should  be  possible  to  determine  what  the  subject’s 
“mental  model”  was  (in  a  global  sense).  This  conceptualization  of  a  “mental  model”  is 
somewhat  broader  than  some  previous  definitions.  It  includes  not  only  explicit  or  implicit 
“awareness”  of  the  present  static  situation,  but  also  the  dynamic  situation.  The  subject  is  nor 
only  modeling  the  static  environment,  but  is  making  behavioral  decisions  based  on  projections 
and  hypotheses  of  “what  is  going  to  happen”  in  the  future.  Thus,  this  measure  attempts  to  tap  the 
strategies  (deeper  cognitive  activities)  being  used  to  translate  present  information  into 
performance. 

Recommended  Techniques  for  Measuring  Information  Requirements/Processing 

Two  relatively  different  techniques  for  assessing  the  person’s  global  “mental  model  are 
possible  in  the  SAFTE  system.  The  first,  directly  related  to  the  work  of  Pew,  Adams,  and 
Tenney  (1991)  and  others,  involves  subjectively  addressing  the  subject’s  awareness  and  action^ 
through  protocol  analysis.  Although  this  capability  currently  exists  in  SAFTE,  it  was  not  ^ 
included  in  the  demonstration  project,  on  the  premise  that  its  application  is  so  mission-specitic 
that  any  resulting  data  would  be  unusable  to  subsequent  investigators.  The  second  technique, 
described  below,  attempts  to  infer  the  subject’s  mental  model  through  explicit  behavioral 
measures. 

Minimum  Information  Flying 


This  technique  is  adapted  from  experimental  approaches  used  by  several  investigators, 
particularly  in  driving  studies,  which  permit  the  subject  to  determine  and  demonstrate  how  much 
information  is  necessary  to  maintain  performance.  To  our  knowledge,  this  technique  has  nevei 
been  suggested  as  an  SA  measure.  In  a  typical  driving  study,  the  subject  drove  an  automobile  in 
traffic,  while  wearing  a  pair  of  occluding  goggles.  Whenever  the  subject  felt  that  additiona 
information  was  required  in  order  to  drive  safely,  he  or  she  was  permitted  to  open  up  the 
occluding  goggles,  which  were  normally  closed.  The  subject  was  instructed  to  minimize  the  time 

the  goggles  were  open,  consistent  with  safety. 

Considering  the  dynamics  of  this  performance,  it  would  appear  that  a  subject’s  decision 
to  open  or  close  the  goggles  is  the  result  of  an  interaction  among  several  factors.  Among  these, 
obviously,  the  complexity  and  demands  of  the  situation  form  the  foundation.  However,  the 
subject’s  understanding  of  the  situation,  the  ability  to  project  the  present  situation  into  the  future, 
the  individual’s  self-perceived  skill  level  to  handle  the  projected  environment,  and  a  variety  ot 
motivational  factors  probably  also  influence  decisions.  Many  of  these  factors,  of  course,  appear 
in  many  definitions  of  situational  awareness.  Therefore,  it  would  seem  appropriate  to  develop  an 
“occlusion  technique”  which  could  be  used  to  ussess  SA. 

Given  the  relatively  standardized  protocols  provided  by  the  SAFTE  system,  it  was  not 
difficult  to  adapt  previous  techniques  in  driving  studies  to  the  SAFTE  system.  Of  course,  in  the 
present  context,  goggles  were  not  necessary.  Instead,  in  the  SAFTE  system,  the  individual  is 
able  to  control  whether  or  not  the  display  is  on  the  computer  screen.  The  person  is  instructed 
that  a  certain  portion  of  the  mission  is  to  be  flown  with  “minimal  information”  consistent  with 
safety  and/or  mission  accomplishment.  The  computer  records  the  time  and  events  occurring 
during  those  periods  in  which  the  individual  sought  information,  and  in  which  the  screen  was 
occluded  or  blank.  Given  that  the  protocols  have  been  developed  specifically  to  demand  certain 
kinds  of  awareness  from  the  person,  the  basic  outcome  of  these  measures  indicates  the  presence 
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or  absence  of  such  awareness. 


Since  the  minimum  information  flying  approach  to  SA  has  never  been  previously  used  in 
such  a  simulation,  certainly  not  in  the  military  context,  a  great  deal  of  study  must  be  carried  out 
before  it  can  reach  the  level  of  parametric  and  statistical  confidence  enjoyed  by  other  measures  in 
the  SAFTE  system.  A  considerable  amount  of  creativity  was  necessary  to  design  research 
protocols  and  flight  scenarios  that  yield  specific  information  about  the  subject’s  SA.  In  addition, 
in  later  studies,  considerable  work  will  be  required  to  assure  that  extraneous  contributors  to  the 
dependent  measures  are  identified  (e.g.,  training  effects,  motivation,  etc.).  On  the  other  hand,  it 
does  appear  that  this  approach,  precisely  because  it  may  capture  a  number  of  such  additional 
determinants,  may  be  particularly  useful  in  assessing  an  individual’s  state  of  training.  In  fact,  it 
might,  in  itself,  have  considerable  training  value  as  the  individual  learns  what  information  is 
really  critical  for  adequate  performance.  In  this  sense,  it  may  be  the  ideal  training  vehicle  for 
increasing  an  individual’s  SA. 

Since  this  is  a  new  attempt  to  measure  situation  awareness,  specific  approaches  to 
analyses  require  considerable  innovation.  During  the  demonstration  study,  various  approaches 
were  considered,  ranging  from  those  that  could  be  objectively  “scored’  to  those  that  approached 
“clinical  interpretation”.  The  basic  dimensions  of  several  of  these  approaches  are  mentioned 
briefly  here. 

1 )  For  a  given  “maneuver”  (e.g.,  a  turn,  a  long  segment  of  straight  and  level  flight,  a 
pop-up,  etc.)  there  are  a  certain  number  of  information  “bits”  that  are  essential. 
Without  these,  the  human  (or  a  machine)  simply  cannot  achieve  the  mission’s  goals. 

If  subject  matter  experts  can  identify  these,  then  a  quantitative  comparison  between 
this  minimum  required  information  and  that  requested  by  the  subject  could  be 
calculated.  Too  little  requested  information  suggests  underestimation  of  data  and/or 
overconfidence,  while  too  much  requested  information  suggests  poor  levels  of 
training,  underconfidence,  or  other  motivational  factors. 

2)  When  a  subject  switches  the  display  “off’,  it  could  be  assumed  that  he  or  she 
believes  that  little  or  nothing  is  going  to  happen  in  the  immediate  future  requiring 
complex  or  visually  directed  action.  From  such  actions,  it  should  be  possible  to 
hypothesize  what  the  subject’s  “mental  model’  of  the  situation  was  at  that  time,  and 
even  the  data  (awareness)  the  subject  had  of  preceding  and  current  situations. 

3)  A  simple  measure  of  effective  “situation  awareness”  could  be  obtained  by  simply 
looking  at  a  ratio  of  the  amount  of  time  the  display  was  turned  off  versus  the 
accuracy  of  flight  and  the  success  of  the  mission.  Clearly,  there  will  be  a  trade-off - 
too  much  time  with  the  display  off  will  certainly  result  in  poorer  mission 
performance.  Too  little  time  may  indicate  lower  SA  capability  (among  other  things). 

This  may  be  the  most  creative  and  innovative  measure  in  the  SAFTE  system.  Hopefully, 
it  will  reveal  aspects  of  SA  that  have  never  before  been  probed.  However,  as  a  new  measure,  it 
will  require  considerable  further  investigation  to  determine  the  best,  most  valid,  and  most  reliable 
approaches  to  analysis  of  the  data.  In  the  demonstration  project  for  this  study,  a  few  of  these 
analysis  approaches  were  attempted,  but  for  reasons  explained  in  Section  Five,  no  extensive 
analyses  were  carried  out. 
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SA  MEASURE  #  6.  SCHEMA  ACQUISITION  AND  DEMONSTRATION 
Rationale  for  the  Measure 


The  final  set  of  measures  that  were  included  in  the  SAFTE  system  involves  the 
development  of  complex  responses  based  on  over-learned  stimulus-response  relationships. 
Various  authors  have  discussed  such  general  response  sequences  under  the  heading  of 
automaticity,  scripts,  or  schemas  (Chase  and  Simon,  1974;  Chi,  Feltovich,  and  Glaser,  1981; 
DeGroot,  1966).  Again,  for  the  present  purposes,  it  is  not  necessary  to  create  fine  distinctions  or 
become  immersed  in  theoretical  discussions  concerning  the  differences  or  levels  of  depth  among 
these  various  conceptualizations. 

The  overall  concept  of  this  type  of  SA  is  seen  most  readily  in  skilled  performance.  There 
is  a  continuum  of  skills  that  is  clearly  recognized  in  training.  Once  the  individual  reaches  a  level 
of  performance  generally  considered  to  be  acceptable  and  safe,  he  or  she  is  already 
demonstrating  behaviors  that  are  radically  different  from  those  of  the  trainee.  While  the  novice 
explores  possibilities  sequentially  and  frequently  considering  alternatives  that  are  clearly 
inappropriate,  the  skilled  performer  appears  to  operate  more  holistically.  A  complex  situation  is 
responded  to  as  a  whole.  In  such  cases,  the  skilled  performer  may  not  even  be  aware  (at  a  truly 
conscious  level)  of  the  elements  making  up  the  whole. 

In  such  cases,  of  course,  probe  techniques  attempting  to  identify  the  individuals  SA  riEiy 
actually  give  a  misleading  impression.  As  the  situation  becomes  more  complex,  and  as  the 
operator  becomes  more  skilled,  he  or  she  may  not  be  able  to  identify  many  specific  elements  in 
the  environment.  The  professional  football  quarterback,  the  karate  expert,  and  the  fighter  pilot 
all  can  respond  to  a  complex  set  of  stimuli  (on  which  they  have  been  well  trained)  with 
incredible  speed.  When  asked  why  they  chose  one  course  of  action  over  another,  they  may  be 
unable  to  identify  all  of  the  elements  (stimulus  discriminants)  in  the  environment  which  triggered 
the  response  pattern,  or  they  may  respond  with  a  “text  book”  answer  having  nothing  to  do  with 
what  they  actually  did. 

This  phenomenon  of  skilled  performance  has  been  discussed  and  debated  at  many  levels. 
With  respect  to  SA,  several  authors  have  discussed  the  role  of  “mental  models”,  defined  as 
“symbolic  representations  of  conceptual  knowledge  that  exist  in  long-term  memory  at  varying 
levels  of  abstraction”  (Cannon-Bowers,  Salas,  and  Converse,  1990).  Harwood  et  al  (1988) 
postulated  that  the  pilot  who  exhibits  good  SA  must  have  an  accurate  mental  model,  and  be  able 
to  update  the  model  as  the  situation  changes.  Endsley’s  (1990)  definition  of  SA  suggests  that  it 
consists  of  the  person’s  mental  model  of  the  surrounding  world,  as  well  as  the  position  occupied 
by  that  person  in  that  world.  She  also  discusses  schemas  as  being  critical  to  SA.  Schemas  are 
defined  as  “memory  stores  which  organize  knowledge  into  integrated  meaningful  frameworks” 
(Shrestha  et  al,  1994).  Fracker  (1988)  explicitly  recognized  that  schemas  are  critical  to  gaining 
an  understanding  of  the  situation  and  triggering  response  patterns.  If  the  incoming  sensory 
information  matches  a  schema  in  long-term  memory,  the  pilot  does  not  have  to  attend  to  evoy 
aspect  of  the  environment  to  have  good  SA.  In  other  words,  the  existence  of  appropriate  schemas 
(and  the  ability  for  them  to  be  activated  by  the  appropriate  match  with  incoming  stimulus 
streams)  constitutes,  in  itself,  a  level  of  SA. 

Approaches  to  Measuring  Schema  Acquisition  and  Demonstration 

In  the  present  case,  the  term  “schemas”  will  be  used  to  refer  to  a  class  of  responses  that 
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are  over-learned,  and  are  based  on  a  frequent  one-to-one  mapping  between  certain  stimulus 
elements  and  certain  response  requirements.  More  generally,  schemas  refer  to  the  individual’s 
characteristic  way  of  responding  to  a  specific,  complex  set  of  environmental  demands.  For 
example,  an  individual  pilot  who  sees  a  specific  formation  of  enemy  aircraft  may  “instinctively” 
carry  out  a  specific  type  of  evasive  or  attack  maneuver.  This  “instinctive”  reaction,  we  believe  is 
really  the  selection  of  one  of  several  potential  schemas  that  have  been  built  up  over  training  in 
the  person. 

The  complement  of  schemas  that  are  available  in  response  to  this  environmental  set  of 
stimuli  may  have  fairly  obvious  stimulus  ’’triggers”,  or  the  triggers  may  be  extremely  subtle.  The 
fact  that  there  are  four  planes  approaching  rather  then  two,  the  fact  that  they  are  spread  a  certain 
distance  apart,  the  fact  that  they  are  of  a  certain  type,  are  all  obvious  determinants  of  schema 
selection.  The  actual  schemas  selected,  however,  may  be  based  on  extremely  subtle  and  maybe 
even  idiosyncratic  stimulus  characteristics  for  that  person.  Subtle,  barely  perceptible  movements 
of  one  of  the  aircraft  may  determine  that  an  evasive  maneuver  rather  than  a  particular  attack 
maneuver  will  be  selected,  or  may  even  determine  the  selection  of  one  of  several  available  attack 
maneuver  schemas.  The  premise  here  is  that  if  the  individual  possesses  his  or  her  “normal”  5A, 
the  response  selected  to  this  complex  stimulus  presentation  will  be  consistent  with  that  which 
was  trained  and  typically  used. 

Under  this  approach,  the  probing  of  the  individual’s  ability  to  select  and  implement  .an 
appropriate  schema  for  a  given  environmental  condition  would  seem  to  be  an  excellent 
assessment  of  the  person’s  SA.  In  those  complex  situations  requiring  the  implementation  of 
schemas,  an  ultimate  awareness  and  sensitivity  of  the  person  to  the  environment  is  demanded. 
This  appears  to  be  clearly  distinct  from  most  of  the  kinds  of  SA  measured  by  the  SA  measures 
proposed  above.  As  such,  therefore,  it  would  appear  that  data  obtained  from  a  probe  of  the 
person’s  implementation  of  schemas  would  be  non-overlapping  with,  or  at  least  complementar/ 
to,  SA  assessments  gathered  in  other  ways.  For  this  reason,  we  have  chosen  to  incorporate 
several  experimental  measures  of  schema  acquisition  and  implementation  into  SAFTE. 

SAFTE’s  Techniques  for  Measuring  Schema  Acquisition  and  Demonstration 

As  an  initial  implementation,  relatively  simple  (and  easy  to  analyze)  situations  involving 
the  development  of  schemas  were  introduced  into  the  SAFTE  system.  A  set  of  four  tactical 
situations  was  developed  involving  air-to-air  engagements  with  various  configurations  of  enemy 
aircraft.  These  scenarios  essentially  involve  minimal  differences  in  the  initial  conditions  (e.g., 
enemy  aircraft  position  and  maneuvers)  which,  nevertheless,  indicate  that  the  pilot  should  take 
radically  different  actions.  For  instance,  a  small  movement  of  one  or  two  approaching  aircraft 
indicates  the  presence  of  fighters  about  to  attack  rather  than  bombers.  Depending  on  the 
subject’s  awareness  of  these  small  discriminants,  the  appropriate  action  might  be  either  to  attack, 
or  to  turn  and  escape.  In  practice,  subjects  would  be  exposed  to  these  differences  many  times  in 
training,  presumably  up  to  the  point  where  appropriate  schemas  would  have  developed.  In  the 
test  mode,  the  pilot  would  be  exposed  to  the  scenario  containing  the  relevant  cue  and  asked  to 
respond  to  it  as  quickly  as  possible.  The  time  to  identify  the  specific  tactical  situation  will  be  the 
primary  dependent  measure.  Exploration  of  this  approach  may  reveal  that  the  operator’s  actual 
response  may  be  a  better  measure  of  SA  than  his  or  her  verbal  report.  In  this  case,  the  measure 
of  SA  could  be  the  actual  piloting  response  to  the  tactical  situation  (e.g.,  change  of  course, 
weapons  enable,  etc.). 

In  this  approach,  therefore,  either  verbal  or  performance  measures  of  SA  could  be 
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obtained.  In  the  case  of  a  verbal  response,  the  response  would  simply  be  categorized  as  either 
correct  or  incorrect  -  meaning  that  either  the  appropriate  response  (schema)  was  or  was  not 
activated.  In  the  performance  response  mode,  classification  can  be  made  not  only  on  whether  the 
appropriate  maneuver  was  made,  but  also  on  specific  measurement  factors  such  as  time-to- 
initiate  the  maneuver  (RT),  and  other  performance-based  measures  of  merit  (Venturino,  et  ai, 
1989). 
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SECTION  FOUR 


PrVEUOPMENT  OF  THE  SAFTE  FLIGHT  SIMULATION  CAPABILITY 

SAFTE  Theory  of  Operation 

This  section  provides  an  overall  description  of  the  configuration  of  hardware  and  software  in  the 
SAFTE  system,  as  well  as  providing  an  operational  description  of  each  of  the  major  portions  of 
the  program.  Detailed  technical  descriptions  of  the  hardware  and  software  requirements  for  both 
the  simulation  and  graphics  machines  are  provided  in  Appendices  1  through  4.  If  the  SAFTE 
system  is  being  hosted  on  hardware  other  than  the  systems  provided,  the  following  sections  on 
requirements  will  need  to  be  reviewed  to  assure  proper  system  configuration. 

Overview 

The  SAFTE  system  runs  on  two  Pentium  class  machines.  One  machine  serves  as  the  simulation 
machine  and  the  other  handles  graphics  tasks.  The  simulation  machine  executes  the  F-16  model 
and  other  simulation  models,  receives  input  from  the  user,  logs  data  and  controls  the  execution  of 
each  scenario.  The  graphics  machine  receives  data  for  its  displays  from  the  simulation  machine 
over  the  parallel  port  and  displays  the  HUD,  instruments  and  the  out-the-window  (OTW)  view. 

The  graphics  machine  should  be  the  fastest  machine  available  to  allow  the  highest  level  of 
graphics  performance.  The  majority  of  simulation  activities  occur  on  the  simulation  machine. 

Hardware  System  Requirements 

Simulation  System  Hardware: 

1 .  1  OOMHz  Pentium  PC  or  better 

2.  VGA  video  board  and  compatible  monitor 

3.  16  Mb  RAM  and  500  MB  hard  disk  space  (or  more  depending  on  data  collection 

requirements) 

4.  Parallel  port  with  bi-directional  capability  (via  CMOS  setup) 

5.  ThrustMaster  Pentium  compatible  game  board  with  trim  pot 

6.  ThrustMaster  controls: 

A.  FLCS  flight  stick 

B.  TQS  throttle 

C.  RCS  rudder 

Graphics  System  Hardware: 

1.  120MHz  Pentium  PC  or  better 

2.  Video  board  supporting  VESA  640x480x256  resolution  and  color  (Number  9  card 

recommended) 

3.  16  Mb  RAM  and  500  MB  hard  disk  space 

4.  Parallel  port  with  bi-directional  capability  (via  CMOS  setup) 

System  Interconnect: 

Shielded  Centronix  style  parallel  port  cable  supplied  by  NTl 


Simulation  System  Hardware 
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The  simulation  hardware  consists  of  a  Pentium  class  PC  running  at  1  OOMHz  or  better,  a 
ThrustMaster  Pentium-grade  game  card,  and  one  set  of  ThrustMaster  controls  as  specified  in  the 
hardware  requirements  section. 

ThrustMaster  Controls 

The  ThrustMaster  controls  are  a  combination  of  digital  and  analog  inputs,  which  are  read  by  the 
interface  software.  The  analog  inputs  (which  include  the  roll,  pitch,  rudder,  and  throttle)  are 
generated  by  creating  a  low  to  high  transition  on  a  control  channel  of  the  control  interface  of  the 
game  card.  A  high  pulse  immediately  appears  at  the  corresponding  input  channel  of  the  game 
card  whose  length  is  proportional  to  the  position  of  the  control  being  sampled.  The  trim  pot 
adjusts  the  range  of  values  of  the  analog  channels. 

The  digital  inputs  are  divided  into  two  classes:  binary  inputs  and  character  inputs.  The  binaiy 
inputs  are  generated  by  user  defined  buttons  on  the  flight  stick  and  are  either  on  or  off  (1/0).  The 
character  inputs  are  generated  by  the  rest  of  the  buttons  on  both  the  flight  stick  and  the  throttle 
and  appear  as  user  defined  ASCII  characters  coming  from  the  keyboard.  Refer  to  the 
ThrustMaster  FLCS  and  TQS  documentation  for  further  information  about  the  controls  and  their 
programming. 

Pentium  PC 

The  PC  on  which  the  SAFTE  program  is  designed  to  run  is  a  generic  Pentium  class  machine  with 
the  addition  of  a  ThrustMaster  game  card.  To  provide  real  time  scheduling  of  events  within  the 
simulation  the  real  time  executive  of  the  software  utilizes  the  hardware  of  the  PC  system  timer  to 
create  a  precisely  timed  clock  pulse  that  is  used  to  sequence  the  real  time  tasks.  Because  the 
system  timer  is  used  by  DOS  to  manage  the  time-of-day  clock,  and  this  function  is  taken  over  by 
the  SAFTE  program.  SAFTE  does  the  management  of  the  time-of-day  clock  while  it  is  running. 

The  operation  of  the  system  timer  is  as  follows:  a  value  from  0  to  65535  is  latched  into  the 
timer's  register  and  the  circuitry  automatically  begins  to  decrement  this  value  to  zero  every  0.84 
microseconds.  On  reaching  zero  a  clock  pulse  is  generated,  the  old  value  is  automatically 
reloaded  into  the  counter  and  the  decrementing  process  resumes.  For  DOS  the  value  loaded 
causes  a  clock  pulse  to  be  generated  at  a  rate  of  1 8.2  Hz.  The  SAFTE  simulation  program  leads 
a  user-specified  value  that  generates  a  clock  pulse  at  a  rate  suitable  for  executing  the  F-16  model. 
Currently  this  rate  is  specified  to  be  50  Hz. 

Graphics  System  Hardware 

The  hardware  of  the  graphics  system  is  also  a  generic  Pentium  class  machine  similar  to  that  of 
the  simulation  system  with  the  following  differences:  First,  the  processor  is  faster,  120  MHz  or 
better,  to  provide  for  the  smoothest  possible  graphics  performance.  Second,  the  video  board  used 
is  of  maximum  performance,  again  to  enhance  the  graphics  display.  Third,  there  are  no  controls 
hooked  into  the  graphics  system  and  thus  no  game  card  is  needed. 

The  essential  function  of  the  graphics  system  is  to  receive  information  from  the  simulation 
system  and  to  display  corresponding  graphical  output  in  as  close  to  real  time  as  possible.  Tc 
provide  as  high  a  throughput  as  possible  all  other  functions  are  carried  out  by  the  simulation 
system. 
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System  Interconnection 

To  get  information  from  the  simulation  computer  to  the  graphics  computer  the  PC's  parallel  port 
was  chosen,  as  it  is  a  built-in  resource  of  adequate  speed.  The  current  implementation  of  the 
parallel  port  software  requires  that  the  parallel  port  be  capable  of  bi-directional  transfer  and  that 
a  custom  cable  (supplied  with  the  SAFTE  system)  be  used.  The  current  effective  transfer  rate  of 
the  interface  in  on  the  order  of  lOK  bytes  per  second. 

Software 

The  purpose  of  the  SAFTE  system  is  to  provide  a  flexible  tool  for  testing  embedded  performance 
and  SA  measures  in  a  real  life  task  (in  this  case  piloting  an  aircraft)  while  running  on  as  close  to 
a  standard  PC  platform  as  possible.  To  accomplish  this  goal  the  software  system  incorporates  the 
following  features: 

1 .  A  realistic  aerodynamic  model  of  the  FI  6  aircraft  having  takeoff  and  landing 
capabilities,  speed  brakes  and  landing  gears.  The  model  is  executed  in  a  real  time 
mode  in  order  to  provide  a  realistic  response  to  the  pilot's  inputs. 

2.  Programmable  SA  probes,  unusual  attitudes  and  screen  blanking  plus  open  ended 
scenario  control  capabilities 

3.  Simulation  objects  such  as  other  aircraft,  radar  threats,  etc. 

4.  A  generic  FI 6  HUD  with  a  radar  warning  receiver  (RWR),  air-to-air  radar  system,  etc. 

5.  Weapons  system  and  countemneasures 

6.  Continuous  logging  of  aircraft  position,  control  inputs,  and  all  pertinent  events 

The  subsystems  of  the  main  software  tasks  (the  simulation  and  graphics  programs)  are  shown 
schematically  in  Figure  1.  The  software  subsystems  are  divided  into  background  and  real  time 
tasks.  Data  generated  by  the  simulation  side,  consisting  of  the  aircraft's  state  vector  and 
navigation  information,  is  transmitted  over  the  parallel  port  to  the  graphics  side,  and  updates  the 
out-the-window  (OTW)  display  as  well  as  the  HUD.  Additional  less  frequent  information  is 
transmitted  over  the  port  in  the  form  of  messages  directed  to  various  systems  on  the  graphics  side 
including  the  RWR  and  air-to-air  displays. 

The  software  system  requirements  are  as  follows: 

Simulation  System  Software: 

1 .  MSDOS  6.2  or  later 

2.  ThrustMaster  Supplied  Software 

3.  DOS4GW.EXE  (Tenbury  via  Watcom) 

4.  CAL.EXE 

5.  SIM.EXE 

6.  SAFTE.BSO  Data  File 

7.  SAFTE  Data  Files  (See  Listing  1  of  the  appendix) 

8.  Files  A.BAT,  M.BAT,  and  S.BAT 
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Figure  1 .  Subsystems  of  the  software  tasks 


Graphics  System  Software: 

1 .  MSDOS  6.2  or  later 

2.  DOS4GW.EXE  (Tenbury  via  Watcom) 

3.  GRAPH1CS.EXE 

4.  Graphics  Data  Files  (See  Listing  2,  of  the  appendix) 

5. FilesG.BAT  and  V.BAT 

Simulation  System  Software 

The  main  functions  of  the  simulation  software  are 

1 .  to  run  the  aircraft  model  and  other  simulation  models 

2.  control  SA  measures  and  other  events 

3.  receive  inputs,  log  data,  and  output  information  to  the  graphics  system 

All  these  activities  plus  several  others  are  executed  within  the  context  of  the  real  time  executive. 

The  executive  (subsequently  referred  to  as  the  RTK)  consists  of  two  main  endlessly  cycling 
loops,  the  background  loop  and  the  real  time  loop.  Within  the  background  loop  are  all  the  tasks 
that  do  not  require  precise  timing  for  their  execution.  The  real  time  tasks  consist  of  tasks  that  are 
time  critical.  The  F-16  model  and  other  aircraft  models  as  well  as  models  of  threats,  bombs, 
missiles,  etc.  fall  into  this  category,  as  they  require  their  execution  to  conform  to  the  passage  of 
actual  or  "real"  time  to  give  a  sense  of  realistic  behavior. 

The  reader  interested  in  more  specific  technical  details  of  the  SAFTE  system  is  referred  to  the 
extensive  descriptions  of  the  SAFTE  system  provided  in  Appendix  1  (Description  of  the  SAFTE 
System  Software),  Appendix  2  (SAFTE  System  Configuration),  Appendix  3  (SAFTE  System  Eiditing 
Tools),  and  Appendix  4  (Instructions  For  the  SAFTE  Simulator  Controls).  For  the  reader  interested 


30 


in  using  the  SAFTE  system  after  it  has  been  set  up,  a  “User’s  Manual”  has  been  developed,  and  is 
included  in  this  report  as  Appendix  5.  This  is  intended  to  guide  the  researcher  in  the  actual  use  of  the 
system,  including  the  kinds  of  data  output  to  be  expected. 


31 


SECTION  FIVE 


INITIAL  SYSTEM  TEST  AND  DEMONSTRATION  PROJECT 
INTRODUCTION 

The  SAFTE  system  represents  an  innovative  development  in  at  least  three  major  areas:  1 )  It 
involves  a  significant  enhancement  of  the  aerodynamic  fidelity  achievable  with  a  PC-based 
system,  2)  it  incorporates  new  techniques  for  assessing  situation  awareness,  along  with  older 
techniques,  and  3)  it  provides  embedded  situation  awareness  measures  in  the  context  of  real- 
world  flight  challenges.  To  achieve  this  product,  an  enormous  amount  of  new  software  had  “o  be 
created,  along  with  significant  developments  in  the  creation  of  realistic  flight  scenarios  and 
implementation  of  actual  situation  awareness  measures.  The  final  deliverable  product  therefore 
represents  a  completely  new  system.  As  such,  the  system  must  be  subjected  to  extensive 
alpha/beta  testing  of  the  hardware  and  software,  as  well  as  the  extensive  validation  and 
psychometric  testing  of  the  SA  metrics.  Obviously,  in  any  ultimate  sense,  such  efforts  in 
themselves  will  take  several  years  and  involve  the  independent  assessment  by  a  large  number  of 
laboratories.  However,  it  was  desirable  and  necessary  to  carry  out  initial  testing  prior  to  final 
delivery  in  order  to  maximize  the  possibility  that  the  delivered  product  will  constitute  the  most 
efficient  and  useable  implementation  of  the  basic  concepts.  This  section  describes  the  first 
system  test  and  demonstration  on  SAFTE. 

The  present  demonstration  effort  had  two  major  goals:  1)  it  was  designed  to  provide  a  rigorous 
test  of  the  software  and  hardware  in  the  initial  SAFTE  design,  and  2)  it  was  designed  to  provide 
initial  and  fundamental  data  on  the  basic  feasibility  and  psychometric  characteristics  of  the  SA 
measures,  as  well  as  the  flight  performance  measures.  With  respect  to  the  first  goal,  the  intent 
was  to  provide  a  “pre-beta”  test  of  the  initial  software  design.  Although  initial  alpha  testing  had 
been  carried  out  during  software  development,  the  system  had  never  been  required  to  perform 
independently  and  interactively.  It  was  inevitable  that  when  such  demands  were  made  on  the 
software,  “bugs”  would  be  uncovered,  and  modifications  would  be  required.  In  the  interest  of 
efficiency  and  economy,  it  was  decided  that  these  activities  would  be  carried  out  in  conjunction 
with  the  testing  of  actual  subjects.  Thus,  the  two  goals  were  integrated  into  one  effort.  In  ef  fect, 
although  it  was  recognized  that  the  data  from  the  early  portions  of  the  study  could  be 
contaminated  by  required  software  changes,  it  was  considered  useful  to  gather  such  data  in  the 
hope  that  at  least  some  of  it  would  be  informative.  In  any  case,  it  was  also  believed  that  the  data 
from  the  later  portion  of  the  study  would  still  be  valid,  and  that  both  goals  would  have  been 
accomplished  in  the  context  of  a  single  study. 

METHODS  AND  PROCEDURES 

The  basic  design  of  this  demonstration  involved  having  civilian  pilots  with  varying  degrees  of 
flight  experience  “train”  on  the  SAFTE  Flight  Simulator.  Such  training  involved  ten  hours  of 
practice  in  a  scripted  set  of  flight  tasks  (see  Appendix  6  for  a  description  of  these  flight  tasks). 
Although  performance  metrics  were  gathered  and  inspected  during  this  period  of  time,  it  was  also 
recognized  that  software  changes  would  be  required  during  the  training.  Therefore,  the  metrics 
could  be  influenced  by  such  software  changes. 

At  the  end  of  the  ten  hours  of  “training”,  subjects  began  the  “testing”  phase.  This  consisted  of  a 
series  of  20  “missions”  which  were  to  be  flown  by  each  subject  (see  Appendix  7  for  a  complete 
description  of  these  tasks.)  The  missions  involved  combinations  of  tasks  that  had  been  practiced 


during  the  training  phase.  During  the  missions,  multiple  examples  of  each  type  of  situation 
awareness  measure  were  presented,  and  these  are  also  detailed  in  Appendix  7.  Flight 
performance  and  SA  measures  were  obtained  and  analyzed  for  this  ten  hours  of  performance. 

The  only  independent  variable  in  this  demonstration  study  was  the  amount  of  flight  experience  of 
the  subject.  Two  levels  of  experience  were  defined  --  0  to  200  hours,  and  over  200  hours.  Only 
total  flight  hours  were  considered  in  this  categorization,  and  no  distinction  was  made  between 
type  of  aircraft  or  training,  since  virtually  all  hours  would  have  been  in  civilian  aircraft. 

Subjects. 

Subjects  were  recruited  through  local  advertising  at  the  Center  for  Aerospace  Sciences  of  the 
University  of  North  Dakota.  Initially,  104  subjects  began  the  training  sessions.  Subjects  were 
paid  by  the  University  at  a  flat  rate  of  $100.00  for  completion  of  the  project.  All  subjects  were 
licensed  civilian  pilots. 

In  view  of  the  University  setting  and  the  rigorous  flight  training  schedule  for  the  subjects,  some 
attrition  of  the  original  subject  pool  was  anticipated.  This  was  especially  true  since  an  attempt 
was  made  to  keep  all  of  the  subjects  on  approximately  the  same  training  point  at  any  given  period 
of  time.  Thus,  if  the  subjects  fell  too  far  behind  in  their  SAFTE  training,  they  would  have  to  be 
eliminated  from  the  study.  During  the  early  portion  of  the  study,  a  slightly  higher  attrition  rate 
was  experienced  than  had  been  expected.  However,  this  was  still  well  within  the  boundaries  of 
original  projections.  However,  midway  through  the  study.  North  Dakota  experienced  a  period  of 
extremely  severe  weather,  with  double-digit  negative  temperatures  for  21  days  straight,  and  a 
series  of  five  blizzards  (eventually  causing  the  University  to  close  and  the  entire  semester  to  be 
canceled).  This  caused  additional  delays  in  both  the  training  and  testing  phases  of  the  effort.,  to 
the  point  that  the  Christmas  holidays  and  mid-year  graduation  occurred  before  the  study  was 
completed.  The  final  result  was  that  very  few  subjects  finished  the  entire  20  hours  of  the  planned 
study.  However,  enough  completed  sufficient  testing  sessions  to  permit  preliminary  statistical 
analyses. 

Scenario  Construction. 

For  both  training  and  test  purposes,  “canned”  scenarios  were  constructed.  These  were  developed 
in  consultation  with  the  pilot  consultant  who  assured  that  they  represented  realistic  missions  and 
flight  parameters.  The  scenarios  described  in  Appendices  6  and  7  were  developed  in  this  way.  A 
total  of  67  training  scenarios  (with  some  scenarios  repeated  in  the  training  sequence)  and  10  test 
scenarios  (each  repeated  twice)  were  developed.  NTl  programmers  hard-coded  these  scenario 
requirements  (e.g.,  the  initiation  point  of  the  training  scenarios  and  the  required  evasive 
maneuvers  of  enemy  aircraft)  in  separate  units  for  presentation  to  the  subject.  To  facilitate  this 
process,  a  “scenario  generator”  was  created  by  NTI  (included  in  the  SAFTE  system  product). 

The  scenario  generator  not  only  facilitated  the  creation  of  the  “hard-wired”  training  and  test 
missions  required  by  the  present  effort,  but  will  permit  researchers  to  generate  their  own  mission 
profiles  to  address  their  research  needs. 

For  the  training  scenarios,  the  sequence  of  “mission  segments”  presented  to  the  students  began 
with  training  on  a  simple  take  off  maneuver,  and  culminated  in  training  on  an  air  to  ground 
attack.  For  each  segment,  the  student  received  a  briefing  card  detailing  the  requirements  for  that 
segment.  In  addition  to  this  card,  the  student  was  given  verbal  instructions  on  the  general  nature 
of  the  simulation,  the  meaning  of  each  HUD  instrument,  and  the  operation  of  all  controls.  This 


was  done  either  by  the  pilot  consultant,  by  the  on-site  experimenter,  or  through  a  video 
instruction.  In  this  latter  case,  instructions  about  how  the  flight  scenarios  should  be  flown  were 
given  through  a  series  of  videos  prepared  by  the  pilot  consultant.  In  these  situations,  upon 
reporting  for  the  session  the  student  would  view  the  five-to-ten  minute  video  pertaining  to  that 
day's  mission.  Each  video  demonstrated  the  maneuver  required  for  that  session  and  provided 
hints  concerning  the  way  the  maneuver  should  be  carried  out. 

For  the  testing  sessions,  a  similar  procedure  was  followed,  except  that,  since  the  subject  would 
already  have  been  familiar  with  each  of  the  mission  segment  requirements,  the  mission 
assignment  was  given  in  a  more  formal  ‘‘briefing”  manner.  Again,  briefing  cards  were 
distributed  to  the  subject,  who  was  given  an  unlimited  amount  of  time  to  read  and  study  them. 
The  subject  retained  these  cards  during  the  testing  session  as  a  kind  of  “knee-pad”  reminder. 
Again,  the  actual  missions  required  of  the  subject,  in  the  sequence  required,  are  presented  in 
Appendix  7,  which  describes  in  detail  the  required  mission  parameters  and  tactics. 

Scenarios  were  presented  in  the  same  order  for  all  subjects.  The  first  ten  testing  missions  were 
all  different  in  either  environmental  condition,  assigned  activity,  threat,  or  required  response. 
The  last  ten  missions  repeated  the  first  ten  in  a  different  order  than  they  had  been  presented 
initially.  Each  scenario  was  designed  to  last  between  1 5  and  25  minutes,  and  subjects  typically 
completed  two  scenarios  per  session.  Scenarios  were  self-administering.  Subjects  were 
instructed  to  attempt  a  scenario  only  once  (i.e.,  if  a  crash  occurred  or  for  some  other  reason  they 
failed  to  complete  the  scenario,  they  were  not  to  repeat  it).  However,  not  all  subjects  followed 
these  directions,  and  some  repeated  scenarios  two  or  more  times  before  proceeding  to  the  next 
scenario.  Thus,  the  training  data  analysis  presented  below  does  not  strictly  reflect  the  same  . 
amount  of  experience  for  each  subject.  Rather,  it  reflects  the  point  at  which  the  subject  felt 
comfortable  at  having  achieved  a  level  of  performance  for  that  particular  training  scenario. 

Situation  Awareness  Measures. 

The  basic  concept  of  the  SAFTE  system  is  to  embed  situation  awareness  measures  into  the  f  ight 
scenarios.  A  total  of  six  different  types  of  SA  measures  were  proposed,  ranging  from  basic 
information  about  what  was  in  the  surrounding  environment  to  an  attempt  at  measuring  the 
development  of  schemas.  For  the  present  demonstration,  instances  of  each  of  these  types  of 
measures  were  developed  by  NTI.  These  were  then  programmed  into  selected  points  in  the 
scenarios.  The  categories  of  SA  measures  and  representative  examples  of  each  have  previously 
been  presented  in  Section  Three  of  this  report. 

RESULTS 


Data  Reduction  and  Analysis. 

Flight  Performance  Measures:  The  first  task  in  analyzing  the  results  of  this  experiment  was  to 
reduce  the  vast  quantity  of  flight  performance  data  collected  to  a  manageable  proportion,  and  to 
select  the  dependent  measures  for  the  flight  task  analysis.  To  achieve  this,  a  “data  generator” 
was  created  as  part  of  the  SAFTE  system  development  (Appendix  8).  This  data  generator 
summarizes  the  continuously  collected  flight  performance  parameters  into  a  series  of  defined 
events  (altitude  change,  heading  change,  weapons  release,  etc.),  and  provides  a  time  stamp  and 
aircraft  state  description  for  each  event.  Specific  variables  presumed  to  measure  the  quality  of 
flight  performance  for  different  maneuvers  were  selected.  A  sample  data  output  for  a  single 
subject  performing  a  single  mission  is  also  shown  in  Appendix  8  for  illustration  purposes.  Once 
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these  maneuvers  had  been  isolated,  along  with  the  metrics  associated  with  each  maneuver,  actual 
data  analysis  could  begin. 


Situation  Awareness  Measures  Data  Analysis.  A  major  focus  of  the  SAFTE  system  is  the 
embedded  SA  measure  technique.  As  noted  above,  there  were  six  different  types  of  SA  measures 
taken  with  different  numbers  of  determinations  for  each  measure.  Interest  primarily  was  in  group 
differences  as  a  function  of  flight  experience.  However,  there  was  also  considerable  interest  in 
changes  in  SA  performance  over  the  course  of  the  ten  hours  of  testing. 

SA  data  were  reduced  in  different  ways  depending  on  the  nature  of  the  SA  probe.  Measures 
using  the  “blanking”  technique  could  all  be  answered  by  a  “yes”  or  “no”  response.  It  was 
therefore  possible  to  determine  a  simple  “percent  correct”  measure.  However,  a  more 
sophisticated  analysis  was  also  possible  due  to  the  fact  that  this  approach  resulted  in  four 
categories  of  response;  “hits”  were  correct  “yes”  answers,  “false  alarms”  were  “yes”  answers 
when  the  correct  answer  was  no,  “true  rejections”  were  correct  “no”  answers,  and  “misses”  were 
“no”  answers  when  the  correct  answer  was  yes.  This  permitted  analyses  to  be  carried  out  using 
information  theoretic  metrics,  as  described  by  Kantowitz  &  Sorkin  (1983)  and  Ambum  (1994). 

Measures  of ‘recoveiy  from  unusual  attitudes’  involved  determining  the  amount  of  time  between 
the  start  of  the  recovery  and  the  point  at  which  the  aircraft  wings  were  straight  and  level  for  over 
three  seconds.  Measures  of ‘minimum  information  flying’  required  individual  analysis  of  each 
segment  of  the  flight.  As  noted  later,  it  became  obvious  early  in  the  test  that  the  ‘schema’ 
measure  was  not  working,  and  therefore  no  analyses  were  carried  out  on  this  measure. 

Flight  Performance  Results. 

There  were  two  major  types  of  analysis  of  interest  of  the  present  data.  One  involved  the  learning 
curves  of  various  measures  for  the  levels  of  flight  experience,  and  the  other  involved  differences 
in  absolute  levels  of  performance.  Both  of  these  questions  were  addressed  by  entering  average 
data  into  two-way  ANOVAs. 

For  this  initial  demonstration,  six  representative  variables  were  selected  for  detailed  analysis,  and 
several  others  were  selected  for  less  detailed  analyses.  These  basic  six  variables  included  the 
following: 

1 :  altitude  at  the  start  of  a  left  turn 
2:  speed  at  selection  of  mil-power 

3:  average  heading  deviation  during  straight  and  level  flight 
4:  average  altitude  deviation  during  straight  and  level  flight 
5:  number  of  straight  and  level  segments  completed 
6:  number  of  straight  and  level  readings 

The  results  of  the  analyses  of  variance  on  these  factors  are  shown  in  Table  1,  and  the  averaged 
data  used  in  these  analyses  are  given  in  Appendix  9.  These  results  are  discussed  more  fully  in 
the  graphic  presentation  of  results  presented  below. 
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TABLE  1 


RESULTS  OF  ANOVA  ON  SELECTED  FLIGHT  MEASURES 

North  Dakota  SAFTE  Demonstration  Analysis 

FMO 1  -  Altitude  at  Start  of  Left  Turn 

FM02  -  Speed  at  Selection  of  Mil-Power 

FM03  -  Average  Heading  Deviation  during  Straight  &  Level 

Flight 

FM04  -  Average  Altitude  Deviation  during  Straight  &  Level 
Flight  . 

FM-A  -  Number  of  Straight  &  Level  Segments 
FM-B  -  Number  of  Straight  &  Level  Readings 
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0.107845 

0.744892 

2 

16 

0.887556 

480 

1.519589 

0.584076 

0.896499 

1x2 

16 

1.82256 

480 

1.519589 

1.199377 

0.264074 

FM04 

1 

1 

17.7105 

30 

121.204 

0.146122 

0.704964" 

2 

16 

83.8089 

480 

109.3112 

0.7667 

0.723872 

1x2 

16 

111.7826 

480 

109.3112 

1.022609 

0.430701 

FM-A 

1 

1 

368.9639 

30 

265.5591 

1.389385 

0.247774 

2 

16 

55.5246 

480 

75.6553 

0.733916 

0.75968 

1x2 

16 

82.209 

480 

75.6553 

1.086625 

0.364805 

FM-B 

1 

1 

1316803 

30 

345865.2 

3.807274 

0.060424 

2 

16 

46028 

480 

59155.1 

0.778091 

0.711109 

1x2 

16 

66392 

480 

59155.1 

1.122332 

0.330669 

1  =  Experience  Level  (<  250  Mrs,  >=250  Hrs) 

2  =  Attempt  Number 
*  p  >  .0500 

**  p  >  .0100 

Considering  first  the  deviation  from  assigned  altitude  at  the  initiation  of  a  left  turn,  Figure  2 
indicates  that  both  low  and  high  experienced  subjects  initially  missed  the  assigned  altitude  by 
300  to  600  feet.  Both  groups  show  a  significant  learning  curve  over  the  first  10  tries.  However 
the  low  experience  group  continues  to  initiate  left  turn  earlier  than  instructed,  and  ultimately  is 
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under-estimating  the  turn  by  almost  200  feet.  The  high  experience  group,  on  the  other  hand, 
generally  levels  off  at  close  to  the  assigned  altitude.  Statistically,  the  learning  curve  is 
significant  at  <.05,  although  the  absolute  differences  between  low  and  high  experience  groups 
never  reached  statistical  significance. 


i 


Figure  2.  Deviation  from  assigned  altitude  for  low  and  higher  experienced  subjects. 

On  other  hand,  Figure  3  on  the  next  page  illustrates  that  deviations  from  the  assigned  speed  at 
selection  of  mil-power  show  significant  differences  between  low  and  high  experienced 
individuals.  In  this  case,  however,  the  low  experienced  subjects  tend  to  be  more  accurate  in  their 
performance  than  the  high  experienced  subjects.  This  effect  is  significant  at  <01 .  In  addition, 
there  is  a  significant  effect  of  attempt  number  at  p<.04,  although  this  appears  to  be  very  variable. 

None  of  the  other  flight  measures  selected  for  analysis  show  statistically  significant  differences 
either  as  a  function  of  practice,  or  between  low  and  high  experienced  subjects.  One  variable, 
however,  shows  a  clear-cut  performance  improvement  with  practice.  This  is  shown  in  Figure  4 
on  the  next  page.  During  the  extremely  difficult  ground  attack  maneuver,  the  deviation  from 
assigned  altitude  for  the  roll-out  showed  significant  improvement  over  10  practice  sessions. 
Although  this  failed  to  reach  statistical  significance,  it  illustrates  the  type  of  changes  that  can  be 
expected  in  the  SAFTE  flight  perfonnance  measures. 

Situation  awareness  measures 

“Blanking”  technique  SA  questions.  Analyses  of  the  SA  questions  presented  while  the 
simulation  was  suspended  were  carried  out  based  on  the  type  of  question  involved.  Type  “A” 
questions  essentially  requested  information  about  surrounding  environmental  conditions.  Tj'pe 
“B”  questions  required  some  projection  to  future  events.  Type  “C”  questions  involved  more 
complex  calculations  of  future  patterns  more  derivative  infonnation  processing.  Again,  these 
were  analyzed  either  in  terms  of  the  percent  correct,  or  in  terms  of  an  information-theoretical  ly 
metric  (d’). 


DEVIATION  FROM  ASSIGNED  ALTITUDE  FOR  INITIATING  LEFT 

TURN 


i— LOW  EXP.  |! 
!-« . HIGH  EXP.  i; 
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DEVIATION  FROM  ASSIGNED  SPEED  AT  SELECTION  OF  MIL- 

POWER 


Figure  3.  Deviation  from  assigned  speed  at  selection  of  mil-power. 


I 


-«k-LOWEXP.  ! 
-a . HIGH  EXP.l 


DEVIATION  FROM  ASSIGNED  ALTITUDE  FOR  ROLLOUT  ON 
GROUND  ATTACK  MISSION 


—♦—LOW  EXP. 
-..-a . HIGH  EXP. 


Figure  4.  Deviation  from  assigned  altitude  during  the  roll-out  (ground  attack). 


With  respect  to  the  percent  correct  metric,  there  was  little  difference  between  the  low  and  high 
experienced  groups,  and  little  difference  between  the  types  of  SA  measures  taken.  During  the 
CAP  missions,  somewhat  more  deviation  is  seen  in  the  type  “C”  measure  between  the  low  and 
high  experience  groups.  Again,  however,  the  low  experienced  group  appears  to  perform 
somewhat  better  than  the  high  experience  group. 

On  the  other  hand,  the  information  theoretic  measure  appears  to  be  more  sensitive  to  the 
differences  between  the  groups.  Figure  5  illustrates  d’  for  all  ground  attack  missions  by 
experience  level.  Whereas  the  percent  correct  metric  showed  little  difference  between  groups 
across  training,  the  d’  measure  shows  interesting  differences.  The  experienced  subjects  show  a 
large  difference  in  d’  on  question  type  “C”,  indicating  better  processing  than  the  high 
experienced  subjects. 
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d’  Measure  of  SA  Over  All  Mission  Types 


type  a  type  B  type  C  •  COMPOS. 

Type  of  SA  Measured 

Figure  5.  d’  measure  for  “blanking”  type  questions  during  ground  attack  missions. 

A  similar  results  was  seen  in  the  analysis  of  all  mission  types  with  the  information  theoretic 
measure.  Again,  low  experienced  subjects  performed  better  on  the  more  complicated  type  “C” 
question  than  high  experienced  subjects. 

“Minimum  information  fiving”  SA  technique.  It  will  be  recalled  from  Section  Three  that  the 
subjects  were  instructed  during  certain  portions  of  the  mission  to  turn  the  displays  off  as  much  as 
possible  consistent  with  safety  and  mission  accomplishment.  For  the  most  part,  these  portions 
were  introduced  during  straight  and  level  flight,  where  the  requirements  were  primarily  to 
maintain  altitude  and  speed,  with  some  navigation.  Some  portions  were  introduced  during  turns 
or  altitude  changes. 

In  view  of  the  fact  that  most  subjects  never  achieved  a  high  level  of  skill  in  flying  the  F-16,  data 
obtained  from  this  measure  during  the  demonstration  are  of  extremely  limited  value.  It  would  be 
relatively  meaningless  to  quantify  the  choices  of  the  subjects,  since  they  were  performing  well 
below  the  level  of  trained  militaiy'  pilots.  For  this  reason,  this  measure  was  not  analyzed  in  the 
present  data. 

Recovery  from  unusual  attitudes  (UA’sl  technique.  As  noted  previously,  the  use  of  unusual 
attitudes  (UA)  to  infer  SA  is  an  indirect  approach.  At  best,  the  inference  is  that  if  the  pilot 
efficiently  recovers  from  UA’s,  he  or  she  can  rapidly  assess  a  spatal  situation,  develop  rapid 
spatial  SA,  and  respond  efficiently.  In  SAFTE,  the  UA’s  are  presented  at  a  discrete  point  in  time 
(after  a  “blanking”  question),  so  it  is  possible  to  determine  rather  precisely  how  long  it  took  to 
recover.  This  is  the  basic  measure  obtained. 

In  the  present  demonstration,  the  civil  aviation  pilots  proved  relatively  incapable  of  recovery 
from  the  unusual  attitudes  presented.  This  was  true  even  though  they  had  been  thoroughly 
briefed  on  recovery  procedures,  and  although  some  of  the  instructor  pilots  regularly  flew  unusual 
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attitude  recovery  practice.  The  speed  and  responsiveness  of  the  F-16  aircraft  appeared  to 
overwhelm  them,  to  the  point  that  few  were  able  to  recover  in  any  time  frame  that  would  provide 
meaningful  data.  For  this  reason,  the  measure  was  not  analyzed  beyond  a  superficial  inspection. 

“Schema”  measures  of  SA.  This  experimental  technique  was  not  successful  in  the 
demonstration.  It  became  clear  during  the  training  that  even  with  the  extensive  practice  provided 
in  the  air-to-air  scenarios,  subject  never  reached  the  level  of  automaticity  that  would  be  necessary 
for  the  measure  to  work.  Therefore,  no  analyses  were  carried  out  on  this  measure. 

Many  other  analyses  were  carried  out  on  the  data  generated  in  this  demonstration.  Most  of  these 
proved  non-significant  statistically,  and  contributed  little  to  the  original  parametric  goals  of  the 
effort.  For  completeness,  however,  figures  showing  results  of  some  of  these  non-significant 
analyses  are  presented  in  Appendix  10. 

DISCUSSION 

Procedural  considerations  in  the  SAFTE  system.  Obviously,  the  sample  of  results  presented 
above  cannot  be  considered  definitive  in  any  sense.  They  are  intended  simply  to  indicate  the 
kind  of  results  that  can  be  generated  by  the  SAFTE  system.  As  such,  they  are  illustrative  of  the 
types  of  analyses  that  might  be  carried  out  by  an  investigator. 

The  principal  outcome  of  this  demonstration  project  was  the  development  of  a  set  of 
recommended  procedures  for  utilization  of  the  safety  system.  During  this  project,  the  majority  of 
procedures  initially  implemented  proved  feasible  and  workable.  However,  there  were  a 
considerable  number  of  lessons  learned  which,  even  at  this  early  stage,  modify  recommendations 
for  future  implementations.  These  positive  and  negative  lessons  are  discussed  below. 

One  of  the  first  things  that  became  obvious  during  this  demonstration  study  was  that  civilian 
pilots,  even  those  with  some  degree  of  experience,  such  as  instructor  pilots,  were  not  able  to 
acquire  the  skills  necessary  to  fly  the  F-16  simulation  over  a  10  hour  instruction  period.  It  had 
been  hoped  that  the  incremental  instructions  given  would  allow  at  least  the  experienced  pilots  to 
perform  at  a  plateau  level  by  the  end  of  training.  This  proved  not  to  be  the  case.  Although  some 
of  the  subjects  did  achieve  an  apparent  plateau  during  the  ten  hours  of  testing,  these  were  too  few 
upon  which  to  base  any  conclusions.  A  first  principle,  then,  is  that  in  any  utilization  of  the  safety 
system  with  non-military,  non-fighter-experienced  pilots,  a  considerable  amount  of  training  will 
be  required. 

On  the  other  hand,  it  should  also  be  noted  that  the  sequence  of  training  materials  used  in  the 
present  demonstration  proved  to  be  effective  in  yielding  a  learning  curve,  even  for  the 
inexperienced  civil  aviation  pilots.  While  plateaus  were  not  reached  frequently,  the  subjects  did 
report  being  comfortable  with  the  progression  from  basic  takeoff  to  complex  air  to  ground 
maneuvers.  It  would  be  expected,  therefore,  that  pilots  having  some  jet  aircraft  experience 
would  be  very  comfortable  with  the  types  of  training  utilized  in  this  demonstration.  It  is 
recommended  that  if  the  subjects  are  experienced  military  pilots,  the  training  sequence  used  here 
and  presented  in  Appendix  6  will  be  completely  satisfactory.  In  fact,  depending  on  the 
experience  level  of  the  pilot,  this  training  could  probably  be  significantly  reduced. 

With  respect  to  the  actual  procedures  used  in  running  this  project,  there  were  some  surprising 
results.  The  safety  software  system  had  been  designed  with  the  intention  that  the  entire 
procedure  would  be  self-administering.  Subjects  were  given  extensive  instructions  about  how  to 


initiate  sequences,  data  logging,  and  other  procedural  details.  The  complexity  of  these 
instructions  was  kept  to  an  absolute  minimum.  Yet,  while  many  subjects  followed  the 
instructions,  a  significant  number  either  found  them  impossible  to  follow,  or  decided  to  ignore 
them.  The  most  common  error  was  for  subjects  to  repeat  missions  on  which  they  felt  they  did 
poorly.  This  clearly  violated  their  instructions.  Other  problems  included  entering  wrong 
identification  numbers,  and  failing  to  log  data  properly.  It  therefore  appears  that  the  procedures 
implemented  in  the  safety  system  to  permit  self-administration  were  not  successful,  at  least  for 
this  subject  group.  Therefore,  for  the  moment,  it  must  be  recommended  that  the  experimenter 
plan  on  monitoring  all  aspects  of  the  study  closely. 

Considerations  in  scoring  the  flight  measures.  In  itself,  the  system  for  data  logging  incorporated 
into  the  safety  system  appeared  to  work  very  well.  The  system  collects  an  extremely  large 
amount  of  data  on  each  flight  and  mission.  In  other  applications,  this  is  proven  to  be  a 
formidable  obstacle  to  analysis  in  many  simulation  systems.  The  existence  of  the  data  generator 
discussed  in  Appendix  8  significantly  alleviated  this  problem  in  the  SAFTE  demonstration.  The 
data  were  summarized  in  an  extremely  usable  fashion,  without  sacrificing  significant  detail.  This 
is  not  to  say  that  data  reduction  in  the  safety  system  is  a  turn-key  affair.  The  data  generator  does 
not  constrain  the  experimenter  to  a  "canned"  approach  to  data  analysis.  Instead,  it  simply 
presents  the  summary  data  in  a  form  that  can  be  manipulated  by  the  experimenter  to  carry  out 
any  statistical  analysis  desired.  Therefore,  it  is  still  necessary  for  the  experimenter  to  determine 
what  aspects  of  the  flight  maneuvers  to  analyze,  create  definitions  of  these  aspects  in  operational 
terms,  decide  on  which  of  the  SA  measures  should  be  analyzed  and  in  what  form,  and  finally  to 
implement  these  analysis  approaches  in  software. 

In  carrying  out  the  above  analyses,  the  experimenter  will  be  significantly  aided  by  the  decisions 
that  have  already  been  made  concerning  definitions  of  specific  maneuvers,  as  described  in 
Appendix  8.  In  practice,  this  tlirned  out  to  be  one  of  the  more  challenging  efforts  of  the  data 
analysis  process,  and  the  existence  of  the  data  generator  now  simplifies  this  problem  for  the 
experimenter.  Detemiining,  for  instance,  when  a  turn  actually  occurred  required  defining  in  a 
quantitative  way  the  differences  between  a  real  turn  and  a  slight  deviation  of  the  airplane  in  a 
lateral  direction.  This  was  done  in  present  case  in  consultation  with  pilot  consultants,  and 
appeared  to  produce  appropriate  classifications  in  this  demonstration  project.  The  experimenter 
is  therefore  free  to  use  these  as  they  currently  exist.  However,  is  still  possible  for  the 
experimenter  to  modify  these  analyses  carried  out  by  the  data  generator  through  relatively  simple 
modifications  of  the  code. 

The  flight  measures  themselves,  even  in  this  limited  demonstration  with  civilian  pilots,  appeared 
to  operate  efficiently.  Learning  curves  were  obtained,  with  some  differences  appearing  as  a 
function  of  training  and  experience.  An  intriguing  result  was  that  lower  experienced  pilots 
tended  to  stay  closer  to  briefed  parameters  than  more  experienced  ones.  This  may  be  due  to  their 
status  as  novice  students,  more  accustomed  to  following  directions,  or  to  some  other  factor.  The 
important  thing  for  the  present  demonstration  is  that  statistically  significant  differences  were 
found  with  the  SAFTE  system. 

SA  measures  analyses.  The  data  output  of  the  first  three  situation  awareness  measures  —  the 
probe  techniques  --  is  extremely  straightforward.  The  percent  correct  answers  for  each  quesi.ion 
are  summarized  by  the  data  analysis  program,  and  can  be  grouped  in  any  desired  way  by  the 
experimenter.  Similarly,  from  the  data  output  it  is  a  relatively  trivial  task  to  formulate  the  tv'o  by 
two  tables  necessary  to  calculate  d'.  In  the  present  demonstration,  the  procedures  recommended 
by  Kantowitz  &  Sorkin  (1983)  were  used,  although  other  computational  formulas  are  available. 
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Scoring  recovery  from  unusual  attitudes  can  be  somewhat  complex.  It  is  necessary  to  establish 
an  objective  criterion  for  when  the  individual  has  recovered  from  the  unusual  attitude.  In  the 
present  case,  the  definition  selected  was  when  the  aircraft  achieved  straight  and  level  flight  for  a 
given  period  of  time,  and  the  primary  metric  was  the  time  from  the  initiation  of  the  unusual 
attitude  to  this  recovery  point.  Again,  this  analysis  becomes  relatively  easy  by  utilizing  the  data 
generator  and  simply  scoring  the  amount  of  time  between  the  two  events.  Other  scoring  criteria 
might  also  be  utilized,  including  time  to  initiate  the  first  correct  recovery  action,  the  number  of 
inappropriate  recovery  actions,  and  analysis  of  altitudes  achieved  during  various  phases  of  the 
recovery  compared  to  ideal  values. 

Clearly,  the  most  difficult  situation  awareness  measure  to  analyze  is  one  included  under  the 
heading  of  "minimum  information  flying"  --  measure  No.  5.  In  the  present  demonstration,  no 
extensive  analysis  of  this  measure  was  carried  out  in  view  of  the  limited  to  value  of  the  data 
obtained  from  commercial  pilot's.  However,  general  concepts  for  how  such  evaluations  might  be 
done  range  from  relatively  simple  approaches  -  e.g.,  determining  when  the  absence  of 
information  causes  a  significant  accident  or  safety  condition  --  to  more  sophisticated  analyses 
dealing  with  the  subject's  mental  model.  This  latter  approach  involves  some  degree  of 
speculation  on  the  part  of  the  experimenter  or  analyst,  since  there  must  be  some  attempt  to  infer 
why  the  subject  felt  that  no  new  information  was  necessary.  However,  this  can  frequently 
involve  a  relatively  straightforward  set  of  assumptions. 
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SECTION  SIX 


INSTALLATION  OF  A  MODIFIED  SAFTE  SYSTEM  IN  A  HUMAN  CENTRIFUGE 

INTRODUCTION 

This  report  details  the  work  NTI  has  accomplished  in  the  revised  statement  of  work.  Task  6. 
SBIR  Contract  No.  F41624-95-C-5008.  It  covers  only  the  work  associated  with  installing  the 
Situation  Awareness  Flight  Training  Evaluator  (SAFTE)  into  the  Brooks  Air  Force  Base  human 
use  centrifuge.  SAFTE  is  a  PC-based,  F-16  flight  simulator  with  performance  and  situation 
awareness  measures.  A  short  description  of  the  status  of  each  subtask  is  provided  below.  The 
last  subtask  was  to  explore  ideas,  beyond  the  currently  funded  effort,  for  future  uses  of  the 
SAFTE  simulator  in  the  centrifuge  environment.  Therefore,  the  narrative  of  this  subtask 
describes  several  desirable  enhancements  to  the  simulator  that  could  contribute  to  the  validity  of 
centrifuge  research. 


OBJECTIVE 

The  primary  objective  of  this  effort  was  to  adapt  the  SAFTE  flight  simulator  to  the  Brooks 
centrifuge  in  order  to  test  the  feasibility  of  using  it  for  piloting  performance  measurement  under 
high  G  loading. 


RATIONALE 

The  basic  SAFTE  system,  when  installed  in  the  Brooks  AFB  centrifuge  will  enable  acceleration 
scientists  to  measure  human  piloting  performance  in  high  G  environments  in  several  planned 
experiments.  Data  collected  will  objectively  determine  the  effectiveness  of  current  and  future 
high  G  protection  equipment  and  techniques.  These  data  will  be  correlated  with  the 
physiological  variables  as  well. 


SUBTASKS 

Sub-task  1  In  this  subtask,  NTI  modified  the  SAFTE  software  to  allow  installation  into  the 
Brooks  human  centrifuge.  The  modifications  included: 

1 .  Interfacing  analog  voltages  from  the  current  centrifuge  control  stick  into  the  SAFTE 
computer. 

2.  Converting  SAFTE’s  HUD  display  of  G  forces  from  the  aircraft  model  into  analog 
voltages  that  control  the  centrifuge’s  rotational  speed  and,  hence,  the  G  forces  on  the 
pilot  or  subject. 

3.  The  development  of  a  primitive  automatic  throttle  positioner  to  maintain  the 
appropriate  aircraft  speed  for  flying  the  various  scenarios. 

Sub-task  2  NTI  reviewed  the  list  of  equipment  selected  by  the  government  to  implement  the 
installation  of  SAFTE  and  other  related  work,  suggesting  changes  where  needed.  NTI  acquired 
all  the  hardware  and  software  products  necessary  to  accomplish  the  objective.  All  hardware  was 
delivered  to  the  appropriate  government  representatives  as  soon  as  possible  for  implementation 
in  the  centrifuge.  All  systems  performed  as  expected. 

Sub-task  3  NTI  worked  with  the  government  scientists  and  engineers  to  provide  mission 
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scenarios  with  high  G  requirements  for  a  demonstration  of  capabilities.  Because  of  the 
efficiencies  of  the  govemment/contractor  team  in  implementing  the  hardware/software  SAFTE 
system  into  the  centrifuge,  NTl  was  able  to  go  beyond  the  task  requirements  and  implement 
additional  concepts  that  originally  were  programmed  for  future  funding. 

1 .  A  simple  seenario  editor  tool  for  designing  specific  centrifuge/subject  flight  profiles 
was  created.  It  creates  visual  cues  the  subject  uses  to  make  turns,  pitch  changes,  and 
rolls  thus  leading  to  controlled  G  maneuvers.  From  the  subject’s  point  of  view,  it 
consists  of  a  “highway  in  the  sky”  out-the-window  display.  By  flying  the  aircraft 
through  “hoops”  the  subject  completes  the  G  maneuvers  required  for  the  specific 
centrifuge  application.  Flying  performance  measurements  are  taken  during  all 
maneuvers.  A  utility  program  combines  the  logged  data  and  the  “hoop”  location 
information  to  generate  an  ASCII  file  with  RMS  error  values,  the  G  forces,  hoop 
number,  segment  number  and  other  information. 

2.  The  subject  actuated  switch  concept  was  also  implemented.  The  switch  is  located  on 
the  force  stick.  Its  activation  is  logged  in  the  data  file  during  the  subject' s  execution 
of  the  mission.  The  latter  allows  the  subject  to  signal  acknowledgment  of  the 
occurrence  of  various  events  as  they  happen.  For  example,  to  indicate  that  he  has 
knowingly  missed  a  turn  or  is  outside  of  expected  parameters  of  flight. 

Sub-task  4  NTl  worked  with  the  government  scientists  and  engineers  to  test  the  installed 
system  and  allow  subject,  near  closed-loop  control  of  the  centrifuge  operation  within  the  bounds 
of  safety.  Although  closed-loop  eontrol  could  have  been  implemented,  the  centrifuge 
engineering  staff  recommended  against  implementing  it  for  two  reasons.  One,  the  lag  of  the 
centrifuge  behind  the  aeromodel  was  within  acceptable  limits  (approximately  150  msec).  Two, 
an  additional  margin  of  safety  is  available  if  true  closed-loop  operation  is  avoided. 

Sub-task  5  During  and  following  a  testing  session,  NTl  staff  and  consultants  met  with 
government  scientists  and  engineers  to  identify  and  provide  concepts  for  future  modifications  to 
SAFTE  for  anticipated  centrifuge  research  needs.  Each  of  these  identified  enhancement  concepts 
is  described  below.  Because  of  the  effieiency  of  the  work,  three  of  the  “future”  concepts  were 
implemented  under  the  current  effort.  They  are  the  design  and  implementation  of  a  simple 
scenario  editor  to  create  flight  specific  profiles,  the  creation  of  a  “highway  in  the  sky”  out-the- 
window  display  with  utility  program  to  integrate  logged  data  with  the  flying  and  centrifuge 
events,  and  the  addition  of  a  subject  actuated  switch  on  the  stick  that  is  logged  in  the  data  file. 

The  eight  concepts  described  below  were  intended  to  span  the  range  from  veiy  practical,  normal 
upgrades  to  advanced  concepts  such  as  training  air  combat  maneuvers  and  tactics  in  the 
centrifuge  rather  than  in  an  airframe.  The  Gantt  Chart  included  after  Concept  8  is  intended  to 
show  the  dependencies  among  some  of  the  concepts. 

Concept  1  -  Auto  pilot  throttle 

The  control  of  SAFTE’s  simulated  F-16  was  designed  for  stick,  throttle  and  rudder  inputs.  Non¬ 
pilot  operators  find  it  difficult  to  fly  on  a  prescribed  flight  path  when  they  must  coordinate  ail 
three  inputs.  Generally,  operation  of  the  simulated  aircraft  requires  stick  and  throttle  command 
inputs  at  a  minimum.  NTI’s  programmer  has  created  a  compensatory  throttle  outside  the  F-16 
aeromodel  that  attempts  to  maintain  a  fixed  airspeed  throughout  missions  involving  turns,  climbs 
and  dives.  However,  a  speed  auto  pilot  that  is  integrated  with  the  F-16  aeromodel  would  provide 
for  smoother  operation  and  enhance  the  quality  of  the  data  obtained  from  the  subject. 
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A  throttle  input  to  the  model  can  be  created  within  Nil’s  aeromodel  that  would  hold  each 
subject’s  flight  through  a  scenario  to  the  same  speed.  If  subjects  are  allowed  to  reduce  their 
speed  through  a  set  of  turns,  climbs  or  dives,  then  the  task  becomes  progressively  easier  and  the 
subject  will  also  be  less  +Gz  stressed. 

WorkPlan 

The  development,  integration  and  testing  of  a  speed  auto  pilot  can  be  completed  as  follows: 

1 .  Define  the  magnitude  of  the  problem: 

test  runs  measuring  the  performance  of  several  subjects  flying  the  hoops, 

measure  the  frequency  of  speed  changes  to  compare  with  same  run  after  modifications. 

2.  Create  throttle  auto  pilot  (Dr.  Stevens,  GTRI) 

3.  Integrate  auto  pilot  with  SAFTE. 

4.  Test  modified  software. 

5.  Compare  data  from  old  and  new  software. 

Period  of  work:  4  weeks 

Concent  2  -  Design  for  coriolis  experiment 

Generally,  +Gz  is  experienced  in  flight  with  changes  to  the  aircraft  that  involve  turn  radii  of 
thousands  of  feet.  In  the  centrifuge,  +Gz  is  generated  by  rotating  the  gondola  containing  the 
subject  around  a  center  point  only  1 5  to  20  feet  from  the  subjects  head.  In  an  aircraft  angular 
motion  is  minimal;  in  the  centrifuge  it  cause  adaptation  of  the  balance  organ  (semi-circular 
canals)  such  that  head  movements  or  speeding  and  slowing  the  centrifuge  can  cause  illusions  of 
coriolis  (cross  coupling)  that  may  affect  operational  performance  measurement.  If  one  uses  the 
centrifuge  to  determine  that  a  modification  to  a  pressure  suit  significantly  improves  flying 
performance,  how  can  one  be  sure  that  it  is  not  just  correcting  some  centrifuge  artifact,  like  the 
coriolis  effect,  rather  than  something  that  is  a  problem  in  an  actual  airplane?  In  other  words,  the 
suit  modification  might  have  no  effect  as  a  counter  measure  in  flying  an  actual  aircraft.  In  short, 
differences  between  centrifuge  and  aircraft  effects  on  performance  need  to  be  understood  to 
properly  interpret  research  findings  from  the  centrifuge. 

NTl  could  assist  the  government  acceleration  scientists  to  design  an  experiment  to  assess  these 
effects.  To  conduct  this  experiment  in  its  entirety,  it  would  be  necessary  to  compare 
performance  data  from  the  centrifuge  with  that  from  an  actual  flight  using  pilots.  This  could  be 
accomplished  with  the  Calspan/SRL  F-16  inflight  simulator.  On  the  other  hand,  an  incomplete, 
but  useful,  experiment  can  be  conducted  entirely  on  the  centrifuge.  Performance  can  be 
compared  at  rest,  at  rest  with  head  motion,  under  +Gz,  under  +Gz  with  small  head  movements 
and  under  +Gz  with  large  head  movements.  The  coriolis  effect  would  be  indicated  by  significant 
differences  between  the  +Gz  with  head  motion  conditions  and  the  at  rest  conditions. 

WorkPlan 

The  contractors  would: 

1.  Write  the  Research  Protocol. 

2.  Validate  all  performance  measures  and  systems  for  correct  operation. 

3.  Reduce,  analyze  and  interpret  the  simulator  data. 

4.  Write  report  of  research  findings  and  present  them  at  the  ASM  A  convention. 

Period  of  work:  12  weeks 
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Concept  3  -  Implement  throttle  input  in  gondola 


To  give  pilots  a  more  realistic  flying  experience  in  the  centrifuge,  a  throttle  needs  to  be  added  to 
the  gondola  for  operation  of  the  F-16  simulator.  With  the  addition  of  the  throttle,  the  centrifuge 
would  become  an  excellent  tool  for  training  air  engagement  tactics  that  would  include  the  G 
force  consequences  of  actions  taken,  Gz  and  some  Gx. 

Workplan 

The  government  would  provide  the  throttle,  mount  it  in  the  gondola  and  provide  the  appropriate 
voltages  to  the  A/D  converter  of  the  simulator  computer.  NTI  would; 

1 .  Interface  the  throttle  to  the  software  through  the  existing  A/D  converter. 

2.  Make  the  throttle  selectable  as  an  option  when  pilots  are  trained  or  tested. 

3.  Create  special  scenarios  for  the  intended  use. 

4.  Modify  the  system  fidelity  for  pilot  acceptance. 

5.  Validate  all  performance  measures  and  systems  for  correct  operation. 

Period  of  work:  8  weeks 

Concept  4  -  A-LQC 

Almost  loss  of  consciousness  (A-LOC)  is  a  condition  in  which  a  pilot  losses  vision  (tunnel  vision 
or  gray  out)  from  high  Gz  onset  that  does  not  result  in  a  loss  of  consciousness.  This  condition  is 
followed  by  the  return  of  vision  with  the  blood  flow  restored  to  the  brain.  A  question  yet  to  be 
answered  is  “how  does  this  condition  affect  pilot  performance  once  vision  is  restored?”  With 
SAFTE  now  in  the  Brooks  AFB  centrifuge,  this  condition  can  be  researched  with  objective 
piloting  performance  measures.  Government  acceleration  scientists  can  spin  subjects  up  to  A- 
LOC  while  they  are  performing  various  missions  and  record  performance  under  various  levels  of 
gray  out.  Performance  can  then  be  correlated  with  the  measures  of  gray  out  to  assess  the 
relationships.  Guidance  on  the  limitations  of  using  A-LOC  as  a  combat  strategy  can  then  be 
formulated  from  the  results. 

Workplan 

NTI  scientists  would  work  with  the  government  providing  recommendations  for  scenario 
changes,  performance  measurement  requirements,  special  training  methods,  data  reduction, 
artifact  removal,  data  analysis  and  data  interpretation.  At  a  minimum,  NTI  would: 

1 .  Assist  in  the  writing  of  the  Research  Protocol. 

2.  Create  special  scenarios  for  performance  measurement  after  A-LOC.  These  may  include 
measures  of  situation  awareness. 

3.  Validate  all  performance  measures  and  systems  for  correct  operation. 

4.  Reduce,  analyze  and  interpret  the  simulator  data. 

5.  Write  report  of  research  findings. 

Period  of  work:  1 0-20  weeks 
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Concept  5  -  Upgrade  of  simulator  to  Windows  95  (F-PASS  Version) 


NTI  is  upgrading  the  flight  simulator  for  application  to  a  drug  certification  test  battery  (F-PASS) 
for  AF.  This  includes  integration  with  the  Windows  95  operating  system  with  eventual  operation 
under  Windows  NT.  The  new  version  uses  Microsoft  Flight  Simulator  terrain  databases  that  are 
modeled  on  actual  geography  throughout  the  world.  In  addition  to  the  realistic  terrain,  the  new 
version  will  contain  aeromodels  for  T-38,  and  T-1  aircraft.  The  simulator  will  also  operate  vv^ith 
other  Windows  applications  if  needed. 

Workplan 

1 .  Purchase  two  CPU  upgrades  ($1000)  and  two  graphics  accelerator  cards  ($250). 

2.  Purchase  two  Windows95  OSs  and  install  them  on  the  centrifuge  and  training  computers. 

3.  Modify  the  F-PASS  version  of  the  software  to  take  its  input  from  A/D  converter  instead  of 
the  game  card.  The  existing  A/D  modules  may  have  to  be  rewritten  in  Direct  X  to  work  in 
the  Windows  95  environment.  The  same  is  true  of  the  D/A  conversion  of +Gz  for  control  of 
the  centrifuge. 

4.  Rewrite  the  graphics  for  the  “highway  in  the  sky”  in  Direct  X  for  Windows  95. 

5.  Port  the  scenario  editor  from  the  DOS  version  to  operate  with  the  new  Windows  95  terrain 
database. 

6.  Validate  all  performance  measures  and  systems  for  correct  operation. 

Period  of  work:  4  weeks 

Concept  6  -  Assessment  of  laser  protection  visor  under  G  forces 

The  new  laser  eye  protection  comes  in  the  form  of  spectacles  that  are  supported  by  the  bridge  of 
the  nose.  Because  of  the  spectacles  light  blocking  qualities,  they  may  distort  some  of  the  HUD 
information.  Two  major  questions  need  to  be  answered: 

1.  During  high  G  turns,  does  the  laser  protection  system  become  displaced  and  affect  mission 
flying  performance?  If  it  does  shift,  how  much? 

2.  Do  laser  protection,  G  forces  and  the  aircraft  HUD  interact  to  degrade  pilot  performance? 

The  first  question  can  be  answered  with  standard  centrifuge  performance  testing  including  the 
use  of  the  centrifuge  flight  simulator.  The  second  question  expands  the  experiment  by  inserl  ing 
an  actual  aircraft  HUD  between  the  subject  and  the  simulator’s  out-the-window  (OTW)  display. 
The  technical  problem  of  this  effort  would  be  to  separate  the  simulator’s  HUD  output  from  the 
display  of  the  instruments  and  the  OTW  scene.  The  simulator’s  HUD  information  would  be 
displayed  through  an  F-16  or  other  aircraft  HUD.  The  remaining  simulator  display  would  use 
standard  graphics  output  for  the  OTW  view  as  well  as  radars  and  instruments.  The  two  questions 
are  summarized  in  the  following  two  research  objectives. 

1 .  Test  the  physical  effects  of  the  new  laser  eye  protection  on  flying  performance  under  G. 

2.  Test  the  readability  of  the  HUD  symbols  with  and  without  the  new  laser  protection  under 
different  G  levels. 

In  both  cases  the  evaluation  criteria  would  be  the  piloting  performance  measures  from  the  F-16 
flight  simulator.  For  both  approaches,  OEO  would  provide  the  laser  eye  protection  system. 
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Workplan  Question  1 
NTI  would: 

1 .  Write  the  Research  Protocol. 

2.  Create  special  scenarios  to  answer  the  research  questions. 

3.  Validate  all  performance  measures  and  systems  for  correct  operation. 

4.  Reduce,  analyze  and  interpret  the  simulator  data. 

5.  Write  the  report  of  research  findings  and  present  them  at  the  ASMA  convention. 

Period  of  work:  1 2  weeks 


Workplan  Question  2 

The  government  would  provide  the  AF  HUD,  mount  it  in  the  gondola  and  provide  the 
appropriate  connections  to  the  output  of  the  simulator  computer.  In  addition  to  the  items  of 
Workplan  1,  the  contractor  would: 

1 .  Interface  the  HUD  to  the  simulator  computer  through  the  appropriate  interface  hardware. 

2.  Make  the  AF  HUD  selectable  as  an  option  when  pilots  are  trained  or  tested. 

Period  of  work:  9-12  Months 

Concept  7  -  Improved  graphics  display 

The  current  simulator  displays  could  be  improved  in  several  ways  to  add  more  realism. 

However,  since  the  projector  system  in  the  gondola  is  currently  at  its  maximum,  a  new  projection 
system  would  have  to  be  purchased  to  implement  any  of  the  following  approaches. 

1 .  Displays  can  be  improved  by  using  a  higher  resolution  graphics  card.  This  approach  would 
improve  the  display  of  the  current  simulation  world  of  SAFTE. 

2.  The  software  could  be  ported  to  an  SGI  computer.  This  approach  would  be  very  expens  ve. 

•  It  would  require  the  purchase  of  an  SGI  computer,  the  A/D  converter  and  D/A  converter. 

•  It  would  require  the  rewriting  of  the  real-time  executive,  the  graphics  calls,  the  A/D  and  D/A 
modules;  in  short,  a  rewrite  of  most  of  the  system. 

Another  version  of  this  approach  would  be  to  add  an  SGI  computer  for  the  graphics  capabilities 
alone.  This  would  require  much  less  expensive  software  rewriting.  However,  there  is  little  room 
in  the  gondola  for  another  computer.  In  our  opinion,  little  would  be  gained  porting  to  an  SGI 
machine  unless  it  would  be  important  for  other  reasons. 

3.  The  displays  would  be  improved  dramatically  by  porting  SAFTE  to  the  Windows95  or  NT 
operating  system  (see  Concept  5)  similar  to  F-PASS.  Because  the  F-PASS  will  use  the 
Microsoft  Flight  Simulator  terrain  databases,  graphics  resolution  will  be  improved  over  i;he 
current  SAFTE  system.  Additional  fidelity  has  also  been  added  to  F-PASS’s  representation 
of  the  cockpit,  instruments,  and  HUD.  This  approach  would  be  the  best  value  because  the 
Microsoft  Flight  Simulator  terrain  databases  are  modeled  on  actual  geography  throughout  the 
world.  A  higher  resolution  graphics  card  would  also  be  required. 

Workplan  for  Approach  3 

1 .  Purchase  a  new  projection  system  with  greater  fidelity  (Sharp  LCD,  Model  3000,  1 024x768 
resolution,  XGA,  super  bright)  and  two  high  resolution  graphics  cards. 


48 


2.  Government  installation  of  all  hardware. 

3.  Complete  all  work  from  Concept  5. 

4.  Validate  all  performance  measures  and  systems  for  correct  operation. 

Period  of  work:  8  weeks 

Concept  8  -  Network  the  SAFTE/centrifuge  system 

The  idea  of  this  concept  is  to  integrate  the  scenario  events  taking  place  within  the  gondola  with 
other  networked  simulation/training  systems  or  models  such  as  those  running  in  the  AESOP  or 
Tac-Air  Soar.  All  systems  operating  in  this  environment  need  to  be  DIS  compliant  at  a 
minimum.  This  allows  each  “player”  to  “see”  a  representation  of  the  other  constructed  or  virtual 
players,  their  movements  in  space  and  their  actions.  It  is  also  possible  to  talk  to  each  othei  as 
well. 

The  basic  work  to  allow  SAFTE  and  the  centrifuge  pilot/subject  to  be  compatible  would  require 
a  basic  network  card  ($1 50.00)  and  considerable  work  on  programming  the  interface.  Assuming 
that  this  would  not  be  attempted  until  SAFTE  was  converted  to  Windows95,  the  emphasis  would 
be  on  creating  compatible  spatial  coordinate  systems.  Windows95  and  NT  contain  the  necessaiy' 
network  drivers  and  Mak  Technologies  (185  Alewife  Brook  Parkway,  Cambridge,  MA  02138, 
(617)  876-8085)  has  ready-to-use  DIS/HLA  interfaces  for  the  PC  ($3500.00). 

The  other  work  required  involves  improving  the  bandwidth  from  gondola  to  the  outside  world. 
The  current  slip  rings  are  noisy  and  bandwidth  limited.  A  senior  design  project  of  students  under 
the  direction  of  Dr.  James  Frazer  entitled  “Improvement  of  the  instrumentation/data  acquisition 
of  the  centrifuge  gondola  using  a  telemetry  system,”  1997,  describes  a  fiber  optic  solution  for 
approximately  $6400.  The  estimated  price  is  low  because  it  does  not  include  thorough  testing  of 
the  installed  products.  However,  the  price  is  a  good  estimate  of  the  hardware  required  and  the 
labor  to  install  it.  Since  most  of  the  testing  would  be  completed  by  government  engineers,  these 
labor  costs  are  not  included  in  this  proposal. 

Should  this  enhancement  to  the  centrifuges  functionality  progress  to  the  level  of  participation  in 
a  large  distributed  simulation  without  the  participation  of  the  AESOP  laboratory,  a  T-1  line 
would  have  to  be  installed  and  supported.  This  communications  link  is  a  very  expensive  to 
maintain.  It  is  recommended  that  any  efforts  involving  distributed  simulation  be  coordinated 
with  the  AESOP  facility  since  it  should  have  a  supported,  secure,  T-1  line  in  the  near  future. 

Workplan  (assumes  Concept  5  has  been  completed) 

1 .  Purchase  a  network  card,  a  fiber  optic  telemetry  system  and  the  development  tool  kit  for  the 
DIS  interface. 

2.  Government  installation  of  all  hardware. 

3.  Program  SAFTE  simulator  system  for  DIS  compatibility. 

4.  Validate  all  performance  measures  and  systems  for  correct  operation. 

Period  of  work:  1 2  weeks 
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APPENDIX  1 


DESCRIPTION  OF  THE  SAFTE  SYSTEM  SOFTWARE 

The  descriptions  of  the  particular  portions  of  the  simulation  software  have  been  broken  down 
functionally  into  input  functions,  processing  functions,  and  output  functions.  Please  note  that 
files,  if  not  explicitly  stated,  lie  within  the  DATA  directory  underneath  the  directory  containing 
the  SIM. EXE  program. 

Inputs 

The  following  are  inputs  to  the  simulation  program.  Some  of  these  inputs  are  only  referenced  at 
initialization  of  the  program  while  others  are  sampled  continuously.  Along  with  each  input 
source  is  a  description  of  the  processing  associated  with  the  input. 

1 .  Configuration  file  (RTCONFIG.DAT)  -  This  is  a  text  file  containing  user  configurable 

parameters.  Of  the  various  parameters  such  as  parallel  and  serial  port  values,  one  of 
the  most  important  is  the  system  timer  value,  discussed  at  the  beginning  of  the 
hardware  section  of  this  document.  The  system  timer's  value  affects  several  aspects  of 
the  simulation  program's  behavior.  The  RTCONFIG.DAT  file  is  read  once  at  the 
beginning  of  the  program. 

2.  ThrustMaster  parameter  file  (TMCAL.DAT)  -  This  text  file  is  automatically  generated  by  the 

CAL.EXE  program  and  specifies  various  parameters  of  the  roll,  pitch,  rudder,  and  . 
throttle  axes  which  are  used  to  characterize  the  control  inputs  for  the  F-1 6  model.  This 
file  is  read  once  at  the  beginning  of  the  program. 

3.  Scenario  database  file  (SCENARIO.DAT)  -  This  file  contains  the  declarations  of  all  objects 

within  all  of  the  scenarios  which  are  available,  including  way  points,  aircraft,  SAM 
sites,  triggers  (see  below),  and  input  files  associated  with  the  scenarios.  Once  a 
particular  scenario  is  specified,  this  file  is  opened  and  the  information  for  the  scenario 
is  read  into  memory.  All  aspects  of  each  scenario  except  the  scenario  control 
instructions  are  contained  within  this  file. 

4.  F16  trim  files  -  The  F16  model,  like  all  true  aerodynamic  models,  requires  its  initial  internal 

state  to  be  properly  configured  and  cannot  just  be  'turned  on'  in  an  arbitrary  state.  To 
accomplish  this,  each  scenario  begins  by  reading  into  the  FI 6  model  a  state  statement 
from  a  trim  file.  Listing  1  of  the  appendix  is  a  trim  file  (TRIM.DAT)  and  various  files 
of  the  form  SCENxx.DAT.  A  SCENxx.DAT  file  (referred  to  in  the  SCENARIO.DAT 
file)  is  read  into  the  TRIM.DAT  file  at  the  beginning  of  a  scenario,  then  the 
TRIM.DAT  file  is  read  into  the  FI  6  model  to  initialize  it's  state. 

5.  Analog  inputs  (pitch,  roll,  rudder,  and  throttle)  -  The  analog  control  inputs  come  from  the 

ThrustMaster  controls  and  appear  at  the  game  card  input  registers  as  signals  to  be 
measured  by  the  software.  As  mentioned  before  in  the  hardware  section  these  control 
inputs  appear  as  pulses  which  are  high  for  a  period  of  time  proportional  to  the  physical 
position  of  the  control.  The  software  converts  this  pulse  length  into  a  normalized  value 
by  timing  the  length  of  the  pulse  then  using  the  parameters  in  the  TMCAL.DAT  file  to 
scale  it.  It  will  be  noted  that  the  analog  input  conversion  takes  place  as  a  real  time 
task.  This  is  for  two  reasons:  First,  it  allows  the  inputs  to  sampled  at  a  precise  rate. 
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Second,  because  software  timing  loops  by  definition  cannot  be  interrupted,  the  task  is 
assured  to  be  accomplished  in  the  real  time  loop  without  being  interrupted. 

6.  Keyboard  and  HOTAS  inputs  -  Keyboard  and  HOTAS  (Hands-On-Throttle-And-Stick) 

inputs  are  in  most  cases  character  inputs  which  are  generated  by  or  come  through  the 
keyboard  and  are  processed  as  they  occur  via  a  keyboard  interrupt  routine.  Valid 
keyboard  entries  are  detailed  in  the  appendix  of  the  "SAFTE  Users  Guide"  document 
and  define  commands  for  events  such  as  way  point  selection,  raising  and  lowering  the 
landing  gears,  and  exiting  the  program.  Controls  from  the  throttle  and  stick  (not 
counting  the  analog  ones  and  certain  exceptions)  define  such  operations  as  HUD  mode 
selection,  speed  brake  engagement,  weapons  release,  and  air-to-air  designate  cursor 
movement.  The  exceptions  are  certain  controls  that  can  be  defined  as  binary  (1/0) 
values  and  are  accessed  through  the  game  card  interface.  These  inputs  are  converted  to 
the  same  format  inside  the  program  as  the  character  inputs. 

7.  Floppy  disk  -  For  applications  where  an  automated  sequence  of  scenarios  is  desired  a  special 

file  named  SAFTE.DAT  residing  on  a  floppy  disk  contains  information  on  the  history 
of  scenario  execution.  When  the  program  is  started  this  file  is  read  and  information 
regarding  the  current  scenario,  scenario  number  and  scenario  history  is  copied  into  the 
program's  memory.  This  information  is  used  both  to  access  the  correct  scenario  in  the 
scenario  database  and  to  create  the  name  of  the  file  on  the  hard  disk  to  which  data  will 
be  logged. 

Processing  of  Scenarios 
Start  of  Scenario 

Processing  within  the  simulation  starts  with  accessing  data  for  the  selected  scenario  from  the 
scenario  database  and  creating  the  data  file  which  will  hold  the  data  taken  during  the  scenario's 
execution.  All  simulation  objects  are  created  and  initialized  and  the  correct  trim  file  read  into  the 
Fi6  model.  All  initial  conditions  specific  to  the  scenario  (such  as  number  of  chaff  rounds, 
landing  gears  up  or  down,  ILS  mode  enabled  or  not)  are  set  and  the  aircraft  is  released  to  fly. 

The  F-] 6  Model 

As  the  scenario  progresses,  inputs  from  the  controls  are  continuously  measured,  scaled  and 
normalized,  and  fed  to  the  aircraft  model.  The  model  takes  not  only  the  roll,  pitch,  rudder,  and 
throttle  values,  but  also  the  value  of  the  speed  brake  and  the  position  of  the  landing  gears  as 
inputs.  The  model  uses  these  to  simulate  takeoffs,  landings  and  free  flight. 

The  output  of  the  model  consists  of  the  output  state  vector.  The  output  vector  provides 
information  on  position,  velocity,  attitude,  angle-of-attack  and  other  parameters.  This 
information  is  made  available  to  the  simulation  system  and  is  also  shipped  across  the  parallel 
interface  to  the  graphics  computer  along  with  other  information. 
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Triggers 


In  order  to  control  events  during  the  course  of  a  scenario  (such  as  launching  enemy  aircraft  or 
putting  up  an  SA  probe  question)  a  device  called  a  trigger  is  used,  so  called  because  its  operation 
is  triggered  by  events  or  conditions  within  the  simulation.  A  trigger  is  a  set  of  user  defined  code 
segments  which  have  the  following  structure.  Each  code  segment  is  open-ended  and  can 
accommodate  an  arbitrarily  complex  set  of  instructions: 

A.  Conditions  -  A  set  of  instructions  which  specify  under  what  conditions  the  trigger  is 
active.  The  Conditions  instructions  return  a  value  specifying  True  if  the  conditions 
are  satisfied  and  False  if  they  are  not.  The  Conditions  instructions  are  evaluated 
continuously  unless  the  trigger  has  been  disabled. 

B.  Initial  Positive  -  A  set  of  instructions  which  are  executed  when  the  Conditions  go 
from  False  to  True. 

C.  Positive  -  A  set  of  instructions  which  are  executed  while  the  Conditions  are  True. 

D.  Initial  Negative  -  A  set  of  instructions  which  are  executed  when  the  Conditions  go 
from  True  to  False. 

E.  Negative  -  A  set  of  instructions  which  are  executed  while  the  Conditions  are 
negative. 

In  addition,  to  the  above  functions,  triggers  can  have  the  following  specifications: 

A.  Execute  the  trigger  a  set  number  of  times  or  repeatedly. 

B.  Evaluate  the  trigger  only  during  a  user  defined  mode. 

C.  Evaluate  the  trigger  only  in  the  vicinity  of  a  certain  region  of  the  terrain. 

D.  Initialize  the  trigger  to  be  active  or  inactive.  Triggers  are  not  executed  or  have  their 
conditions  evaluated  unless  they  are  activated. 

Simulation  Objects 

There  is  currently  a  limited  set  of  simulation  objects  supported  by  the  program.  These  objects 
are  entities  created  at  the  beginning  or  during  the  course  of  a  scenario  which  can  interact  with  the 
pilot  and  with  each  other  in  non-intelligent  or  semi-intelligent  ways.  These  objects  include  the 
following: 

A.  The  pilot's  aircraft  -  This  object  includes  the  F-1 6  model  along  with  speed  brakes, 
landing  gears,  and  other  instrumentation.  The  pilot's  aircraft  appears  to  the 
simulation  like  any  other  object  in  the  simulation. 

B.  Other  aircraft  -  Semi  intelligent  objects  possessing  radar  systems  and  weapons. 

C.  SAM  sites  -  Semi  intelligent  threat  centers  capable  of  launching  SAMs. 

D.  AAA  sites  -  Semi  intelligent  threat  centers  like  SAM  sites. 

E.  Bombs  -  dumb  objects  created  and  launched  by  the  pilot's  aircraft  or  other  aircraft. 

F.  SAMs  and  AAMs.  These  objects  can  be  created  and  launched  by  SAM  sites  (for 
SAMs)  or  by  the  pilot  and  other  aircraft  (for  AAMs). 

Currently  the  pilot,  other  aircraft,  and  SAM  sites  are  the  only  objects  being  used  in  the  scenarios 
created  to  date.  Support  exists  for  the  other  objects  both  in  the  simulation  and  graphically  on  the 
graphics  side,  though  some  work  still  remains  to  make  all  objects  fully  functional. 
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Instrumentation 


Along  with  the  objects  of  the  simulation  is  the  set  of  simulated  instruments.  The  instrumentation 
list  includes  the  following: 

A.  Radar  Warning  Receiver  (RWR)  -  This  instrument  detects  hostile  radar  sources  and 
displays  them  on  the  RWR  screen.  Each  type  of  threat  has  it  own  symbology  and 
lethality  parameters.  When  a  new  threat  is  detected  by  the  RWR  system  a  'new  guy' 
signal  sounds  alerting  the  pilot  of  the  threat's  presence  and  symbology  appears  on  the 
screen  placed  in  the  direction  of  the  threat  and  at  a  distance  from  the  display's  center 
based  on  the  threat's  calculated  lethality.  When  no  new  threats  are  being  processed 
the  RWR  emits  a  tone  corresponding  to  the  threat  with  the  greatest  lethality  of  the 
currently  detected  threats. 

B.  Air-to-Air  (AT A)  Radar  -  The  ATA  radar  displays  returns  of  other  detected  aircraft 
relative  to  the  pilot's  current  heading.  The  track  ball  and  mouse  button  on  the 
throttle  allow  movement  of  the  ATA  cursor  and  designation  of  a  target  to  launch 
against.  The  symbology  of  the  ATA  radar  includes  the  relative  heading  and  altiuide 
of  all  detected  aircraft.  For  a  designated  aircraft  additional  information  is  displayed 
at  the  top  of  the  display  which  includes:  target  aspect,  target  heading,  ground  speed, 
and  closure  rate.  Additionally,  new  symbology  appears  on  the  ATA  screen  and  the 
HUD  screen  indicating  the  missile's  minimum  and  maximum  ranges  and  the  time  to 
activation  of  the  missile  once  the  missile  is  launched. 

C.  ILS  system  -  During  the  time  the  landing  gears  are  down  the  ILS  symbology  is 
displayed,  if  enabled.  The  ILS  system  consists  of  a  simulated  ILS  beacon  at  the  end 
of  the  airport  and  simulated  equipment  onboard  the  aircraft  which  directs  the  pilot  to 
the  approach  glide  slope  of  the  runway.  The  ILS  equipment  consists  of  an  angle-of- 
attack  (AOA)  bracket,  ILS  deviation  bars,  and  a  flight  director.  The  deviation  bars, 
and  the  flight  director  symbology  can  be  enabled  or  disabled  based  on  the  scenario 
designer's  desires.  Figure  1  of  the  appendix  describes  the  various  symbols  of  the  ILS 
system. 

D.  Other  instruments  -  The  remaining  instruments  include  the  following: 

1 .  Speed  Brakes  -  located  just  under  the  HUD  mode  switch,  the  speed  brake  switch 
extends  the  brakes  proportionally  to  the  amount  of  time  the  switch  is  pulled 
toward  the  pilot,  up  to  a  maximum  of  60  degrees  (45  if  landing  gears  are 
engaged).  Pushing  the  switch  away  retracts  the  brakes  to  zero. 

2.  Weapons  release  -  The  red  button  at  the  top  of  the  flight  stick  releases  bombs 
and  launches  missiles.  If  in  AAM  mode  a  'time  to  active'  message  will  be 
displayed  on  the  ATA  radar  screen  once  a  missile  is  launched. 

3.  Counter  measures  -  Chaff  and  flares  can  be  selected  by  toggling  back  and  forth 
with  the 't'  key  on  the  keyboard.  The  paddle  switch  at  the  base  of  the  flight  sitick 
releases  two  (2)  bundles  of  chaff  or  two  (2)  flares  each  time  it  is  pressed. 

Outputs 

Output  from  the  simulation  program  goes  to  one  of  three  places:  the  screen,  the  parallel  port  and 
the  hard  disk/floppy  disk.  Of  the  three  destinations  the  parallel  port  is  the  most  heavily  used, 
followed  by  the  hard  disk,  then  the  screen  and  floppy  disk.  Following  is  a  description  of  the 
information  content  of  each  path. 
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Screen  -  The  screen  is  the  least  used  path  mainly  because  most  of  the  user  interface  comes  from 
the  graphics  side  of  the  system;  nevertheless,  the  simulation  side  must  communicate  to  the  user 
such  things  as  queries  for  information  and  notice  of  scenario  completion.  The  simulation  side  is 
also  where  the  scenario  is  initialed  by  typing  "SIM"  at  the  DOS  prompt. 

Floppy  disk  -  For  applications  where  an  automated  sequence  of  scenarios  is  executed  the  floppy 
disk  contains  information  about  the  execution  history  of  the  scenarios.  At  the  end  of  each 
scenario  this  information  is  updated  in  preparation  for  the  next  run. 

Hard  disk  -  At  the  beginning  of  each  scenario  a  file  is  opened  with  a  unique  name  on  the  hard 
disk  in  preparation  for  data  logging.  As  data  is  logged  it  is  recorded  in  the  file  according  to  the 
formats  specified  in  Diagrams  2a  and  2b  of  the  appendix.  When  the  file  is  first  opened  an  LDF 
record  is  written  to  the  file.  Subsequent  data  records  (OUTvect,  INPUTvect,  CHARvect,  Savect) 
are  then  written  preceded  by  an  LDH  record  which  specifies  the  type  and  size  of  the  record 
immediately  following  it.  For  more  information  on  data  logging  see  the  appendix  here  and  in  the 
"SAFTE  Users  Guide"  document. 

Parallel  Port  -  The  parallel  port  data  can  be  broken  down  into  three  types  occupying  three 
sections  in  the  parallel  port  data  vector:  aircraft  output  vector,  navigation/ILS  information  and 
messages.  Note  that  the  contents  of  the  data  vector  is  transferred  at  a  preset  rate  as  defined  fiy 
the  system  timer  and  information  in  the  RTM.DFN  file  and  is  not  driven  by  the  data  itself  (see 
diagram  3  in  the  appendix  for  details  on  the  .DFN  files). 

A.  F-1 6  output  vector  -  This  vector  contains  the  complete  output  vector  from  the  F-1 6 
model  and  is  passed  in  its  entirety  to  the  graphics  machine.  The  appendix  of  the 
"SAFTE  Users  Guide"  document  details  the  contents  of  the  output  vector. 

B.  Navigation/ILS  information  -  This  information  contains  data  on  the  position  of  the 
current  way  point  and  target  point  as  well  as  ILS  symbology  data.  This  data  is  sent 
this  way  and  not  as  a  message  (see  next  item)  because  it  must  be  continuously 
updated  and  displayed.  Diagram  4  of  the  appendix  gives  details  of  the  data  vector's 
contents. 

C.  Messages  -  This  is  a  section  where  messages  going  to  the  RWR,  the  ATA  radar,  the 
instrument  displays  and  other  destinations  are  written.  The  messages  are 
multiplexed  through  the  message  area  with  each  message  having  its  own  ID  nurriber 
and  sub  ID  number  which  is  used  within  the  parallel  port  interrupt  routine  on  the 
graphics  side  to  route  the  message  to  its  correct  destination.  Another  routine  on  the 
graphics  side  scans  the  different  message  destinations  and  keeps  the  graphics  system 
updated  with  the  latest  information. 

Scenario  End 

A  scenario  can  be  ended  in  one  of  several  ways: 

A.  The  ending  conditions  are  reached  or  the  pilot  lands  the  plane  successfully. 

B.  The  pilot  crashes  the  plane. 

C.  The  pilot  runs  out  of  time  in  the  scenario  (the  maximum  time  of  each  scenario  is 
programmed  into  the  scenario). 

D.  The  pilot  aborts  out  of  the  scenario  for  some  reason. 

E.  The  computer  hangs  or  crashes. 
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Each  of  these  endings  is  reflected  in  the  name  of  the  data  file  when  it  is  updated  at  the  end  of  the 
scenario.  The  appendix  of  the  "SAFTE  Users  Guide"  document  lists  the  ending  values  for  each 
condition  and  the  format  of  the  data  file  name. 

Graphics  System  Software 

The  graphics  program  is  only  responsible  for  the  creation  and  presentation  of  graphics 
information.  No  simulation  control  or  processing  of  inputs  takes  place  in  the  graphics  compjter, 
as  that  is  the  function  of  the  simulation  computer. 

The  primary  functions  of  the  graphics  system  are: 

A.  Display  of  the  Out-The-Window  (OTW)  view. 

B.  Display  and  update  of  the  F-1 6  HUD 

C.  Display  of  other  instrumentation  (speed  brakes,  power  setting,  etc.) 

D.  Display  of  messages  (SA  probes,  etc.) 


Inputs 

Several  inputs  to  the  graphics  program  are  similar  to  those  of  the  simulation  program  while 
others  (e.g.  the  terrain  database)  are  unique  to  this  system.  Following  is  a  description  of  eaci  of 
the  input  sources.  If  not  explicitly  specified,  all  input  files  are  to  be  found  in  the  DATA  sub  ¬ 
directory: 

A.  RTC0NF1G.DAT  -  This  file  does  for  the  graphics  system  exactly  what  it  does  for 
the  simulation  system;  it  provides  parameter  information  for  the  RTK  exec,  the  most 
important  value  being  the  system  timer  value. 

B.  MESSAGE.DAT  and  .TXT  files  -  These  file  are  for  the  display  of  messages  such  as 
SA  probes.  The  MESSAGE.DAT  file  contains  a  list  of  all  message  files  used  by  the 
scenarios.  The  .TXT  files  contain  the  actual  messages. 

C.  Miscellaneous  .DAT  files  -  Files  like  SAM. DAT,  AC. DAT,  etc.  are  data  files  used 
for  displaying  various  objects  in  the  OTW  view. 

D.  Keyboard  inputs  -  The  graphics  system  does  not  rely  on  keyboard  input  to  the  degree 
that  the  simulation  system  does.  The  major  keyboard  interactions  are  to  execute  the 
program  GRAPHICS.EXE'  from  the  DOS  command  line  and  to  quit  the  program  via 
the  'Q',  'q',  ’X’,  'x',  or  'ESC  keys. 

E.  Aircraft  output  vector  -  The  output  vector  of  the  aircraft  is  shipped  to  the  graphics 
side  at  a  constant  rate  determined  by  the  system  timer  value  and  the  contents  of  the 
RTM.DFN  file  on  the  simulation  side.  Within  the  output  vector  are  values  needed  to 
drive  the  HUD  display  and  the  OTW  view.  Each  receipt  of  a  new  parallel  port 
transmission  engages  the  parallel  port  interrupt  handler  which  transfers  the  output 
vector  to  it's  proper  place  in  memory. 

F.  Navigation/ILS  data  -  Within  the  data  vector  transmitted  during  each  parallel  port 
transmission  is  information  on  way  points,  targets,  and  ILS  displays.  This  data  is 
used  to  drive  additional  instrumentation  located  on  and  off  the  HUD. 

G.  Messages  -  At  the  end  of  the  parallel  port  data  vector  is  an  area  devoted  to  message 
transmission.  To  this  area  is  written  data  designated  for  different  systems  on  the 
graphics  side.  Each  packet  of  data  contains  a  major  and  a  minor  ID  which  tells  the 
graphics  software  which  system  the  data  is  targeted  for.  When  a  parallel  port 
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transmission  is  received  the  interrupt  handler  on  the  graphics  side  decodes  the 
destination  of  any  existing  message  and  routes  the  data  to  it's  appropriate  place. 

Data  sent  in  this  fashion  is  necessarily  multiplexed  as  there  is  only  one  message  area. 
This  quite  acceptable  for  data  which  is  infrequent  in  nature. 


Processing 

As  in  the  simulation  system,  the  RTK  presides  over  the  execution  of  the  real  time  task  list  and 
the  background  task  list.  One  significant  difference  between  the  two  systems  is  the 
predominance  of  background  tasks  in  the  graphics  system.  The  only  real  time  task  currently  on 
the  graphics  side  is  the  maintenance  of  signals  used  for  blinking  various  displays. 

The  graphics  processing  can  be  divided  into  the  following  sequence  of  operations.  This  is  th  e 
order  in  which  data  is  received  by  the  system  and  converted  into  graphics  displays: 

A.  Output  vector  and  message  parsing  -  Messages  and  the  contents  of  the  aircraft  output 
vector  are  processed  by  the  parallel  port  interrupt  handler  and  distributed  to  thei:* 
respective  memory  locations.  Within  the  background  processing  is  a  routine  called 
Update  which  processes  the  received  information.  Update  looks  for  new  data  and 
executes  code  which  utilizes  the  data.  Examples  of  this  kind  of  processing  include 
SA  messages,  RWR  and  ATA  radar  target  lists,  and  starting  and  stopping  the 
graphics  processing. 

B.  OTW  view  -  After  messages  have  been  parsed  the  graphics  processor  proceeds  to 
create  the  out-the-window  view.  The  OTW  view  includes  the  following  sequence  of 
drawings: 

1 .  Sky  -  The  color  of  the  sky  is  set  according  to  the  conditions  of  the  scenario: 
day/night,  clear/cloudy. 

2.  Moon  and  stars  -  If  the  scenario  is  a  night  mission  the  moon  and  stars  will  be 
drawn  on  top  of  the  sky. 

3.  Horizon  -  The  OTW  processing  first  displays  an  infinite  horizon  on  which  is 
drawn  the  rest  of  the  scenery.  The  color  of  the  horizon  is  derived  from  the  color 
of  the  playing  field  terrain. 

4.  Terrain  -  The  ground,  including  all  flat  objects  such  as  rivers,  roads  and  airf  elds 
but  not  including  mountains  is  drawn  on  top  of  the  horizon  and  sky. 

5.  Mountains  and  all  non-flat  objects  including  light  sources  are  next  drawn  on  top 
of  the  flat  terrain.  Processing  is  done  in  this  order  due  to  the  use  of  a  modified 
version  of  the  painter's  algorithm  which  is  fast  but  subject  to  certain  limitations. 

C.  HUD  -  The  HUD  symbology  is  driven  by  the  contents  of  the  aircraft  output  vector. 
As  new  data  is  transmitted  over  the  parallel  link  the  HUD  display  is  updated  to 
reflect  the  changes.  The  HUD  is  drawn  on  top  of  the  OTW. 

D.  RWR  -  The  radar  warning  receiver  (RWR)  is  displayed  along  with  the  ATA  radar 
display  but  after  the  HUD.  Information  for  the  RWR  comes  from  a  list  of  targets 
transmitted  to  the  graphics  side  in  a  message  once  every  three  seconds. 

E.  ATA  radar  -  The  air-to-air  radar  is  displayed  along  with  the  RWR  display  but  after 
the  HUD.  Information  for  the  ATA  radar  comes  from  a  list  of  targets  transmitted  to 
the  graphics  side  in  a  message  once  every  three  seconds.  Included  in  the  ATA 
display  is  the  relative  position  of  each  return  along  with  it's  altitude.  Designated 
targets  are  circled  and  additional  information  is  displayed  at  the  top  of  the  display 
including  target  aspect  angle,  heading,  ground  speed,  and  closing  rate.  Additionally, 
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a  missile  minimum  and  maximum  range  scale  appears  for  launch  purposes  both  in 
the  ATA  display  and  on  the  HUD.  When  a  missile  is  launched  the  time  to  activation 
is  displayed  at  the  bottom  of  the  ATA  display. 

F.  Cockpit  and  instruments  -  The  cockpit,  represented  by  a  gray  background  behind  and 
between  the  RWR  and  ATA  radar  displays,  contains  various  instrumentation 
displays  including  speed  brake  position,  throttle  position,  landing  gear  status  and 
countermeasures  information.  The  data  for  most  of  these  displays  comes  from  one 
or  more  messages  sent  from  the  simulation  side. 

G,  Text  messages  -  When  SA  probes  or  other  messages  are  to  appear  on  the  graphics 
screen  a  message  is  sent  over  from  the  simulation  side  specifying  which  message  to 
show  and  how  it  is  to  be  displayed.  The  message  itself  is  stored  on  the  graphics  side 
in  the  DATA  sub-directory  and  an  index  of  all  available  messages  is  stored  in 
another  file  in  the  same  directory  by  the  name  ofMESSAGE.DAT.  The  number  of 
the  message  is  used  to  look  up  in  MESSAGE.DAT  the  file  name  of  the  text  message. 
After  getting  the  file  name  the  file  is  opened,  the  graphics  screen  is  blanked,  and  the 
text  of  the  message  is  displayed. 

System  Interface 

The  software  interface  consists  of  two  interrupt  handlers  which  utilize  a  custom  handshaking 
procedure  to  transfer  data  across  the  interface.  When  data  is  ready  to  be  transferred  from  the 
simulation  system  to  the  graphics  system,  the  simulation  system  signals  the  graphics  handler  and 
causes  the  graphics  system  to  be  interrupted.  The  graphics  system  then  signals  the  simulation 
system  that  it  is  ready  and  causes  the  simulation  system  to  enter  it's  interrupt  handler  as  well. 

When  both  systems'  handlers  are  active  another  level  of  handshaking  is  initiated  and  data  is  ihen 
transferred  byte  by  byte  from  the  simulation  side  to  the  graphics  side.  At  the  completion  of  the 
transfer  further  handshaking  signals  the  end  of  the  transfer  and  both  system  exit  their  handlers. 
Currently  only  transfers  from  the  simulation  side  to  the  graphics  side  are  supported. 

The  various  Listings,  Figures,  and  Tables  on  the  next  several  pages  will  aid  the  reader  in 
understanding  the  text  presented  above. 
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BKG.DFN 

RTCONFIG.DAT 

RTM.DFN 

SCEN1.DAT 

SCEN2.DAT 

SCEN25.CUR 

SCEN25.DAT 

SCEN3.DAT 

SCEN37.DAT 

SCEN40.DAT 

SCEN49.DAT 

SCEN5.DAT 

SCEN55.DAT 

SCEN7I.DAT 

SCEN75.DAT 

SCEN99.DAT 

SCENARIO.DAT 

TMCAL.DAT 

TRIM.DAT 

UNUATTD1.DAT 

UNUATTD2.DAT 

UNUATTD3.DAT 

UNUATTD4.DAT 

Listing  1.1 

Simulation  Data  Files 


AAA.DAT 

AC.DAT 

BKG.DFN 

BOMB.DAT 

MESSAGE.DAT 

MISSILE.DAT 

1VISG????.TXT 

RTCONFIG.DAT 

RTM.DFN 

SAM.DAT 

SCENAR77.TXT 

STARS.DAT 

TERRAIN.DAT 

Note:  7  =  wildcard 


Listing  1.2 
Graphics  Data  Files 


Flight  Path  Marker  -  ILS  Beam  OK 


Flight  Path  Marker  -  No  ILS  Beam 


Horizontal  ILS  Dev.  Indicator 


Vertical  ILS  Dev.  Indicator 


Angle  of  Attack  Bracket 


Figure  1.1 
ILS  Symbols 
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td_TCNT:  //  Timer  Count  Value  Record 


w: 

unsigned  integer,4  bytes 

//  Timer  Count  Value 

point; 

X: 

Y: 

//  "Point"  Record 
integer,  2  bytes 
integer,  2  bytes 

//  X  coordinate 
//  Y  coordinate 

Rtim:  //  RTK.  System  Time  Record 

cyc_num:  unsigned  integer,  4  bytes 

frame_num:  integer,  4  bytes 

subframe_num:  integer,  4  bytes 

TMRcnt:  td_CNT,  4  bytes 

//  Cycle  Number 
//  Frame  Number 
//  Sub-frame  Number 
//Timer  Count  Value 

EventRecord: 

what: 

message: 

linkcnt: 

when: 

where: 

modifiers: 

//  RTK  Queue  Record 
integer.  4  bytes 
integer,  4  bytes 
integer,  4  bytes 

Rtim,  1 6  bytes 
point,  4  bytes 
integer,  4  bytes 

//  Event  Type 

//  Type  Dependent  Information 
//  Event  Link  Byte  Count 
//  RTK  Sys  Time  since  RTK  launch 
//  Mouse  Location  at  Event 
//  Keyboard  Modifier  Flags 

TStamp: 

esecs: 

Tract: 

//Time  Stamp  Record 
unsigned  integer,  4  bytes 
fioat.  4  bytes 

//  Elapsed  Seconds 
//  Fractional  part  of  seconds 

Diagram  1.1 

Basic  Data  Record  Types 


LDFhdr: 

//  Log  Data  File  Header  Record 

frame_mult:  unsigned  integer,  4  bytes 

//  Frame  Multiple 

timer_cnt:  unsigned  integer,  4  bytes 

//Timer  Count 

LDBhdr: 

//  Los  Data  Block  Header  Record 

type:  integer,  4  bytes 

//  LogData  Block  Header  Type 

cnt:  integer,  4  bytes 

//  #  of  bytes  to  follow  in  Block 

OUTvect: 

//  Aircraft  Output  Vector  Record 

type:  integer,  4  bytes 

//  Type  of  Buffer 

t:  Tstamp,  8  bytes 

//  Time  at  Data  Write 

Erec:  EventRecord,  36  bytes 

//  Time  at  Data  Write 

X[25]:  float,  4x25  bytes 

//  Aircraft  Data  Vector 

INPUTvect:  //  Controls  Input  Vector  Record 

type:  integer,  4  b>tes 

//Type  of  Buffer 

t:  Tstamp,  8  bytes 

//  Time  at  Data  Write 

Erec:  EventRecord,  36  bytes 

//  Time  at  Data  Write 

X[4]:  float,  4x4  bytes 

//  Controls  Input  Vector 

CHARvect;  //  Character  Data  Record 

type:  integer,  4  bytes 

//  Type  of  Buffer 

t:  TStamp,  8  bytes 

//  Time  at  Data  Write 

Erec:  EventRecord,  36  bytes 

//  Time  at  Data  Write 

X:  char,  1  byte 

//  Character  Data 

SAvect: 

//  SA  Data  Vector  Record 

type:  integer,  4  bytes 

//  Type  of  Buffer 

t:  TStamp,  8  bytes 

//Time  at  Data  Write 

Erec:  EventRecord,  36  bytes 

//  Time  at  Data  Write 

X[2]:  integer,  4x2  bytes 

//  Aircraft  Data  Vector 

Diagram  1.2 

Data  Logging  Record  Formats 
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[Procedure  Name]  [Frame  Mask]  [Module  Mask] 

Frame  Mask  -  16  bit  number,  each  bit  of  which  if  set  to  1  enables  the 
procedure  for  that  frame 

Module  Mask  -  16  bit  number,  each  bit  of  which  if  set  to  1  enables  the 
procedure  for  that  module  type 

Note:  One  frame  is  one  clock  pulse  of  the  system  timer  interrupt 


Diagram  1.3 
.DFN  File  Format 
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Aircraft  Output  Vector 
(See  SAFTE  Users  Guide  appendix  for  contents) 
Start  Byte  Stop  Byte 

1  100 

Bearing  to  Target  or  Way  Point 
Start  Byte  Stop  Byte 

101  104 

Distance  and  Time  to  Way  Point 
Start  Byte  Stop  Byte 

105  108 


Current  Way  Point  Position  / 

Slant  Angle  for  CCIP  Mode 
Start  Byte  Stop  Byte 

109  120 

Current  Target  Position 
(AAM  &  CCIP  Modes) 

Start  Byte  Stop  Byte 

121  132 

Current  Pipper  Position  for  CCIP  Mode  / 
Slant  Range  of  Desigated  Target  / 

ILS  Deviation  Bar  Data 
Start  Byte  Stop  Byte 

122  136 


Slant  Angle  of  Desigated  Target  / 

ILS  Flight  Director  Data 
Start  Byte  Stop  Byte 

137  144 


Start  of  Message  Buffer  Area  / 
Message  ID 


Start  Byte 

Stop  Byte 

145 

148 

Message  Buffer  (Continued) 

Start  Byte 

Stop  Byte 

149 

264 
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Current  Target  Position 
(AAM  &  CCIP  Modes) 

Stop  Byte 
132 

Current  Pipper  Position  for  CCIP  Mode  / 
Slant  Range  of  Desigated  Target  / 

ILS  Deviation  Bar  Data 
Start  Byte  Stop  Byte 

133  136 


Diagram  1.4 
Data  Vector  Contents 


Start  Byte 
121 
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Message  Name  JD 


Start/Stop  1 

Number  of  Graph  Objects  2 
SAM  Site  Object  3 

AAM  Site  Object  4 

Enemy  Aircraft  Object  5 

SAM/AAM  Object  6 

Bomb  Object  7 

RWR  Display  8 

ATA  Radar  Display  9 

Instrument  Displays  10 

Text  Message  1 1 

HUD  Mode,  etc.  12 


Table  1.1 
Message  Types 


[X  Position]  [Y  Position]  [Text  Me; 

Coordinate  Ranges  Min.  Max. 

X  Position  0  639 

Y  Position  0  479 


Table  1.2 

Text  Message  Format 
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APPENDIX  2 


SAFTE  SYSTEM  CONFIGURATION 


The  SAFTE  system  physically  consists  of  two  Pentium  computers  connected  to  each  other  via 
the  parallel  port  of  each  machine.  Due  to  the  computationally  intensive  natures  of  the  F-16 
aerodynamic  model  and  the  out-the-window  graphics  the  graphics  functions  are  run  on  one 
machine  and  the  aircraft  and  scenario  simulation  on  another.  For  effective  real  time  system 
performance  the  graphics  machine  runs  at  120  MHz  or  better  with  16  MB  of  RAM  and  VESA 
640x480x256  color  graphics  support.  The  simulation  machine  runs  at  90  MHz  or  more  with  8 
MB  of  RAM  and  at  least  540  Mbytes  of  hard  disk  space  to  store  data  logged  during  scenario 
execution.  The  peripheral  equipment  consists  of  one  of  each  of  the  following:  ThrustMaster 
throttle,  rudder,  flight  stick,  and  game  board. 

SAFTE  provides  continuous  time  stamped  disk  logging  of  control-  input  data  at  5  Hz,  aircraft 
state  vector  information  at  1  Hz,  and  keyboard  and  other  pertinent  data  as  they  occur.  To  achieve 
sub  millisecond  time  stamping  accuracy  in  real  time  a  custom  real  time  kernel  (RTK)  operating 
system  provides  both  precise  event  timing  and  access  to  system  resources.  With  the  RTK  re.il 
time  operation  of  the  F-16  aerodynamic  model  as  well  as  other  simulation  entities  is  provided 
for. 

The  aircraft  state  vector  consists  of  29  different  quantities  including  the  position,  attitude, 
velocity  and  angle  of  attack  (AOA)  of  the  aircraft.  The  control  inputs  include  roll,  pitch,  rudder, 
and  throttle  values.  Other  inputs  include  keyboard  and  hands  on  control  entries. 

Along  with  pilot  inputs  various  events  are  also  logged,  including  SA  queries,  simulation  control 
events  and  radar  activity. 

The  continuous  time  stamped  logging  of  the  above  data  items  forms  a  small  but  essential  set  of 
raw  information  from  which  a  wide  variety  of  statistics  can  be  computed. 


Use  of  Utilities  for  the  SAFTE  Centrifuge  Project 


The  programs  comprising  the  SAFTE  centrifuge  project  are  as  follows: 

1 .  SIM.EXE  -  The  simulation  program. 

2.  PROFlLE.EXE  -  A  program  for  generating  flight  paths. 

3 .  HlSTORY.EXE  -  A  report  generating  program  for  data  analysis. 

The  purpose  of  these  programs  is  to  allow  the  user  to  exercise  subjects  using  predefined  flight 
paths,  collect  data  from  the  subjects,  and  create  reports  in  a  format  suitable  for  reading  or  import 
into  a  spreadsheet  or  other  analysis  tool.  Following  are  details  on  the  operation  of  each  program. 

SIM.EXE 

The  SIM  program  is  the  main  simulation  utility.  The  simulator  contains  an  FI 6  model  along 
with  a  HUD  and  other  instruments  on  the  screen.  Once  the  necessary  preliminary  data  is  entered, 
the  simulator  is  started  and  controlled  by  the  force  stick  and  its  button  inputs.  Gz  is  sent  to  the 
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centrifuge  to  drive  it's  operation  and  data  is  collected  during  the  flight  which  is  written  to  the 
hard  disk  at  the  end  of  the  run.  The  simulator  is  exited  by  pressing  <ESC>  on  the  keyboard 

To  access  the  simulation,  start  the  simulation  computer  and  at  the  C:\  prompt,  type  "z"  and 
<enter>.  This  places  the  user  in  the  main  simulation  directory.  It  is  very  important  to  follow  this 
step  and  to  not  simply  change  to  the  simulation  directory  instead.  Several  important  path 
assignments  are  made  via  this  procedure.  The  startup  command  "z"  may  be  changed  as 
necessary. 

To  begin  the  simulation,  type  "sim  s"  and  <enter>.  The  user  will  be  prompted  to  enter  an  ID 
number.  This  number  is  stored  in  the  file  IDS.DAT  in  the  SAFTEDAT  sub-directory  and  can  be 
changed  by  the  user.  Currently  the  numbers  "123"  and  "999"  reside  there. 

After  the  ID  is  entered  the  program  will  prompt  the  user  for  a  scenario  number.  To  execute  the 
predefined  flight  paths  enter  "97".  The  screen  will  display  a  message  stating  the  purpose  of  the 
scenario  and  wait  for  a  key  to  be  pressed.  Once  a  key  press  occurs  the  simulation  begins. 

Following  are  the  functions  supported  by  the  simulation  and  how  to  access  them; 

Pitch  and  Roll  -  Accessed  by  pressure  on  the  force  stick. 

Throttle  -  Pushing  the  center  lever  up  increases  power,  pushing  down  decreases  power. 
Landing  gears  -  The  right  button  on  the  force  stick  toggles  the  gears,  as  well  as  'T  on  the 
keyboard. 

Trigger  Button  -  The  front  button  on  the  force  stick  is  for  specialized  user  input. 

Way  Points  -  The  left  button  on  the  force  stick  toggles  to  different  way  points. 

Recording  -  The  "r"  key  on  the  keyboard  toggles  recording  of  flight  path  status  -  See 
PROFILE  below. 

Exit  -  the  <ESC>  key  on  the  keyboard  terminates  the  program  and  saves  all  collected  d2.ta. 

NOTE:  the  Caps  Lock  on  the  keyboard  must  be  OFF  for  proper  operation  of  controls. 

Once  the  simulation  has  ended,  control,  status  and  performance  data  are  stored  in  a  uniquely 
named  file  in  the  main  simulation  directory.  The  first  two  characters  of  the  file  name  represent 
the  scenario  number  and  the  file  name  extension  represents  the  subject  ID. 

PR0F1LE.EXE 

The  PROFILE  program  is  used  for  generating  predefined  flight  paths.  These  flight  paths  are 
presented  to  the  subject  as  a  "rope  in  the  sky"  consisting  of  "hoops"  to  fly  through.  The  flight 
paths  can  consist  of  combinations  of  five  styles: 

1 .  Straight  -  A  line  in  space  for  a  given  number  of  seconds  at  the  current  speed. 

2.  Left/Right  turn  -  A  level  turn  from  the  current  heading  at  a  given  Gz  value  and  for  a 
given  number  of  degrees. 

3.  Up/Down  turn  -  A  vertical  turn  from  the  current  altitude  at  a  given  Gz  and  for  a 
given  number  of  degrees. 

The  flight  path  is  generated  from  a  profile  stored  in  a  user  created  file.  Each  line  in  the  file 
contains  a  command.  The  file  starts  with  the  key  word  "start"  and  ends  with  the  key  word  "stop". 
Following  is  an  example  profile  illustrating  the  syntax  of  the  commands: 
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start 

straight  10.0 
left  2.0  90.0 
straight  10.0 
right  3.0  90 
straight  15.0 
up  2.0  45.0 
down  2.0  90.0 
up  2.0  45.0 
straight  10.0 
stop 

This  file  would  generate  a  profile  consisting  of  10  seconds  in  a  straight  line,  a  left  turn  at  2  Gs 
for  90  degrees,  a  straight  run  for  1 0  seconds,  a  right  turn  at  3  Gs  for  90  degrees,  straight  for  :  5 
seconds  up  at  2  Gs  to  45  degrees,  over  and  down  at  2  Gs  to  -45  degrees,  an  up  turn  at  2  Gs  tC' 
level  off  and  a  final  straight  segment  for  10  seconds. 

Before  running  the  PROFILE  program  a  trim  file  must  be  created  for  the  aircraft.  This  tells 
PROFILE  what  the  starting  speed,  altitude  and  attitude  of  the  aircraft  will  be.  To  create  this  file 
start  the  simulation  and  execute  the  scenario  "99".  This  is  a  generic  scenario  allowing  the  user  to 
fly  the  plane  to  the  position  desired  for  the  start  of  the  profile.  When  the  starting  conditions  are 
achieved  toggle  the  recording  mode  of  the  simulation  by  pressing  the  "r"  key,  wait  1  -  2  seconds, 
then  press  it  again.  The  necessary  data  will  have  been  recorded  in  the  correct  file. 

After  the  trim  data  has  been  generated,  type  "profile  <filename>"  from  the  main  simulation 
directory  where  <filename>  is  the  name  of  the  user  created  profile  file.  The  newly  generated 
data  will  be  placed  in  the  correct  file  for  future  use. 

If  needed,  a  complete  profile  set  can  be  archived  by  backing  up  the  user  created  profile  file  and 
the  files  HISTORY.TRM  and  HlSTORY.DAT  in  the  DATA  sub-directory.  Note  that  each  profile 
generation  overwrites  the  contents  of  these  two  files  so  backing  up  these  files  is  necessaiy'  if  the 
profile  is  to  be  saved  for  future  use. 

HlSTORY.EXE 

The  HISTORY  program  generates  a  report  file  from  the  generated  profile  data  and  the  collected 
simulation  data.  HISTORY  attempts  to  match  each  recorded  position  of  the  aircraft  with  the 
nearest  hoop  in  the  prerecorded  flight  path.  The  report  file  is  in  text  format  and  includes  on  each 
line  the  following  information: 

1 .  Time  into  scenario 

2.  Miss  distance  from  the  hoop 

3.  Type  of  hoop  (straight,  left  turn,  up,  etc.) 

4.  Set  number  of  hoop  (first  straight  set,  second  left  turn  set,  etc.) 

5.  Index  into  the  current  set  (first  hoop,  second  hoop,  etc.  in  this  set) 

6.  Gz  value  of  the  hoop 

7.  Event  value  -  currently  represents  Trigger  value  (1  -  pressed,  0  -  released) 

To  execute  the  HISTORY  program  type  from  the  main  simulation  directory: 


69 


history  <report  name>  <data  file  name>  [<profiIe  data  file  name>] 

The  report  name  is  the  name  of  the  report  file.  The  data  file  name  is  the  name  of  the  simulation 
data  file  and  can  be  found  by  examining  the  contents  of  the  main  simulation  directory.  The 
profile  data  file  name  is  optional  and  allows  a  custom  profile  data  file  name  and  path  to  be 
entered. 

SAFTE  DATA  LOG  DESCRIPTION 

To  understand  the  contents  of  the  data  log  files  some  explanation  of  their  generation  should  oe 
given.  When  any  program  constructed  using  the  Real  Time  Kernel  (RTK)  process  is  executed, 
the  RTK  replaces  the  PC’s  system  timer  interrupt  (INT  8)  with  it’s  own  interrupt  routine.  While 
this  allows  the  real  time  processing  of  user  defined  tasks  it  also  means  that  the  DOS  time-of-day 
clock  is  disabled  and  thus  the  DOS  clock  will  be  retarded  by  the  length  of  time  the  program  is 
running  (unless  some  provision  is  made  to  update  the  clock  during  or  after  execution). 

The  PC’s  timer  counts  down  at  a  rate  of  1 1 93 1 80  Hz.  To  use  the  timer  a  value  from  0  to  65535 
is  loaded  into  the  timer’s  register.  The  timer  automatically  begins  to  count  down  and  when  a 
value  of  zero  is  reached  an  interrupt  is  generated  and  the  previous  value  is  reloaded  into  the 
counter.  This  behavior  gives  rise  to  a  regular  clock  interrupt  which  for  a  counter  value  of  65535 
occurs  at  a  rate  of  1 8.2  Hz,  the  familiar  DOS  system  timer  frequency. 

Under  the  RTK  system  the  counter  value  (and  thus  the  timer  frequency)  becomes  a  user-defined 
quantity  allowing  the  fine  control  of  real  time  processes.  For  the  current  SAFTE  system  both  the 
aero  software  and  the  visual  software  are  programmed  (via  the  file  RTCONFIG.DAT)  to 
generate  a  system  clock  interrupt  at  a  50  Hz  rate.  The  value  of  the  counter  along  with  some 
additional  data  is  recorded  in  the  data  log  file. 

Logging  Record  Types 

There  are  three  classes  of  records  recorded  in  each  data  file. 

These  are: 

A.  LogData  file  header  records.  These  contain  a  number  known  as  the  frame  multiple 
and  the  counter  value  used  to  generate  the  desired  frequency  of  the  timer  interrupt. 
See  LOG.H  for  details. 

B.  LogData  block  header  records.  These  records  indicate  the  type  and  size  of  the  next 
record  to  follow.  See  LOG.H  for  details. 

C.  Data  records.  These  records  contain  the  actual  simulation  data.  These  records  are 
further  divided  into  four  types  as  explained  in  the  next  section.  Each  data  record  is 
preceded  by  a  block  header  record. 


Data  Types 

There  are  four  types  of  data  recorded  under  the  SAFTE  system. 

1 .  OREC  (typedef  OUTvect)  -  output  vector  from  the  aero  model.  The  definition  of  the 
vector  values  is  found  in  the  file  F16.H. 

2.  IREC  (typedef  INPUTvect)  -  control  inputs  from  the  pilot.  This  is  an  array  of  size 
four.  The  array  elements  in  order  are: 
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ROLL,  PITCH,  RUDDER,  THROTTLE. 


3.  CHREC  (typedef  CHARvect)  -  character  from  the  keyboard  or  ThrustMaster 
controls.  See  related  information  on  control/keyboard  input  definitions. 

4.  S AREC  (typedef  SAvect)  -  general  vector  of  SA  information.  This  vector  is  for 
recording  general  information  usually  relating  to  SA  data.  The  vector  has  two 
elements:  a  type  field  and  a  value  field.  The  type  specifies  the  type  of  SA  data  being 
collected  and  the  value  records  the  quantity  being  collected.  See  DPLUSR.H  for 
definitions  of  the  different  SA  types. 

Data  Record  Formats 

All  four  types  of  data  records  have  a  common  format  as  seen  from  their  definitions  in 

FILTERS. H.  This  format  is  as  follows: 

A.  Type  -  type  of  record  (OREC,  IREC,  CHREC,  SAREC) 

B.  Timestamp  -  another  record  recording  the  record’s  generations  in  seconds  and 
fractions  thereof. 

C.  EventRecord  -  a  record  comprised  of  subrecords  which  records  system  information 
as  well  as  specific  message  information.  The  event  record  is  used  here  to  provide 
timing  of  the  record’s  generation  to  the  nearest  tick  of  the  clock  (0.838  microsecs.). 

D.  Data  Vector  (or  Scalar)  -  the  last  item  in  the  record  is  the  actual  data  being  recorded. 
The  data  types  have  been  covered  in  the  previous  section. 

Data  Recording  Sequence 

Each  type  of  data  record  is  logged  under  it’s  own  specific  conditions.  These  conditions  are  as 

follows: 

A.  ORECs  -  The  aero  state  vector  is  record  every  second.  The  time  between  ORECs 
can  vary  by  a  few  percent. 

B.  IRECs  -  The  control  vector  is  recorded  five  (5)  times  per  second.  Like  the  ORECs 
the  IRECs  may  vary  a  few  percent  in  inter-record  timing. 

C.  CHRECs  -  Characters  are  recorded  as  generated  so  are  asynchronous. 

D.  SARECs  -  SA  events  are  recorded  as  generated  so  are  asynchronous. 

OUTPUT  OF  THE  F1LTER.EXE  PROGRAM 

To  illustrate  the  above,  the  following  text  is  a  excerpt  of  the  output  of  the  filter.exe  program  as 

run  on  an  actual  data  file: 


71 


I!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 
frame  mult:  1  <—  dividing  factor;  1  =  execute  every  interrupt 

timer  count:  23864  <---  timer  count  value  23864  =  50  Hz 

TS  ET  TY  DATA 

T:  1.33  (1.33),  SAREC:  (0(SIM_MESSAGE),  71)  <—  SA  type:  message,  data:  msg  71 
T:  1.44  (1.46),  IREC:  P:  0.8  R:  -5.9  TH:  1 15.7  RDR:  9.7  <-  Control  Inputs 
T:  1.60(1.62),  IREC:  P:-1.5  R:  -6.9  TH:  115.7  RDR:  11.1 
T:  1.84(1.86),  IREC:  P:  0.0  R: -6.9  TH:  115.7  RDR:  11.1 
T:  2.00  (2.02),  P:  (0.00,  0.00,  0.00) 

T:  2.00  (2.02),  IREC:  P:  0.8  R:  -5.9  TH:  1 15.7  RDR:  9.7 
T:  2.24  (2.26),  IREC:  P:  0.8  R:  -5.9  TH:  1 16.5  RDR:  12.5 
T:  2.40  (2.42),  IREC:  P:  0.8  R:  -5.9  TH:  1 15.7  RDR:  9.7 

T:  2.57  (2.57),  SAREC:  (4(SIM_T1ME),  1)  <—  SA  type:  sim  time  toggle,  data:  start 

T:  2.64  (2.66),  IREC:  P:  0.8  R:  -5.0  TH:  1 15.0  RDR:  9.7  T:  2.80  (2.82),  IREC:  P:  0.0  R:  -6.9 
TH:  115.0  RDR:  11.1 

T:  3.04  (3.06),  P:  (30.98,  134.19,  2.59)  <—  position  (NORTH,EAST,ALTITUDE) 

T:  16.77  (16.77),  CHREC:  I  (108)  <—  I  =  raise/lower  landing  gear 

T:  23.98  (24.00),  SAREC:  (0(SIM_MESSAGE),  7101)  <—  message:  msg7101 
T:  23.98  (24.00),  SAREC:  (0(SIM_MESSAGE),  7101)  <—  repeat 
T:  23.99  (23.99),  SAREC:  (4(S1M_T1ME),  0)  <—  stop  sim  time 

T:  25.96  (25.98),  CHREC:  H  (72)  <—  slider  switch  “yes”  response  to  probe 
T:  25.97  (25.97),  SAREC:  (4(SIM_TIME),  1)  <—  start  sim  time  again 
T:  26.12  (26.14),  CHREC:  L  (76)  <—  return  of  slider  switch  to  middle  pos. 

T:  35.20  (35.22).  CHREC:  w  (1 19)  <—  toggle  waypoint 

T:  35.20  (35.22),  SAREC:  (7(SIM_CUR_WP),  0)  <-  value  of  new  WP  (=  0) 

T:  35.44  (35.46),  IREC:  P:  12.5  R:  19.6  TH:  84.5  RDR:  9.7 

T:  35.52  (35.50).  CHREC:  w  ( 1 19)  <—  toggle  waypoint 

T:  35.52  (35.50),  SAREC:  (7(SIM_CUR_WP),  1)  <—  value  of  new  WP  (=  1) 


NOTES: 

TS  =  Timestamp  of  data  from  the  data  record 

ES  =  Elapsed  time  calculated  from  the  EventRecord  data  in  the  data  record 

TY  =  Type  of  record  (IREC,  OREC,  etc.) 

data  =  data  within  the  record  (ORECs  are  partially  listed) 

IMPORTANT:  It  will  be  noted  that  certain  SARECs  are  of  the  type  S1M_T1ME. 

Because  the  simulation  can  be  “frozen”  at  different  times  a  record  of  the  starting  and  stopping  ot 
“sim  time”  must  be  provided  in  order  to  calculate  the  elapsed  time  in  the  simulation.  The 
timestamp  of  data  is  based  on  the  elapsed  time  of  the  RTK  and  is  independent  of  the  time  in  the 
simulation. 

MISCELLANEOUS  NOTES: 

A.  All  keyboard  input  is  in  lower  case,  all  control  button  input  is  upper  case.  Refer  to  the 
file  SAFTE.B50  to  view  the  programming  of  the  control  buttons. 

B.  Each  data  file  begins  with  a  LogData  file  header  record. 

C.  Each  data  record  is  preceded  by  a  LogData  block  header. 
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NOTES; 

TS  =  Timestamp  of  data  from  the  data  record 

ES  =  Elapsed  time  calculated  from  the  EventRecord  data  in  the  data  record 
TY  =  Type  of  record  (IREC,  OREC,  etc.) 

DATA  =  data  within  the  record  (ORECs  are  partially  listed) 

IMPORTANT:  It  will  be  noted  that  certain  SARECs  are  of  the  type  S1M_T1ME. 

Because  the  simulation  can  be  “frozen”  at  different  times  a  record  of  the  starting  and  stopping  of 
“sim  time”  must  be  provided  in  order  to  calculate  the  elapsed  time  in  the  simulation.  The 
timestamp  of  data  is  based  on  the  elapsed  time  of  the  RTK  and  is  independent  of  the  time  in  the 
simulation. 

MISCELLANEOUS  NOTES: 

A.  All  keyboard  input  is  in  lower  case,  all  control  button  input  is  upper  case.  Refer  to  the 
file  SAFTE.B50  to  view  the  programming  of  the  control  buttons. 

B.  Each  data  file  begins  with  a  LogData  file  header  record. 

C.  Each  data  record  is  preceded  by  a  LogData  block  header. 
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APPENDIX  3 


SAFTE  EDITING  TOOLS  GUIDE 


This  document  describes  the  tools  available  for  creating  and  modifying  both  the  scenarios  ar,d 
graphics  of  the  SAFTE  system.  These  tools  were  developed  for  the  initial  generation  of  scenario 
and  graphics  databases  and  as  such  should  be  viewed  as  a  work  in  progress.  Some  features  of  the 
tools  are  inoperative  due  to  a  continuing  state  of  flux  in  the  conceptualization  of  the  database  and 
some  require  a  full  development  environment  (i.e.,  compiler)  to  be  used.  This  document  will 
detail  the  use  of  the  tools  as  they  presently  exist. 

Scenario  Generator 

The  scenario  generator  provides  for  the  creation  and  modification  of  scenarios.  A  scenario 
defines  the  starting  point  of  the  pilot's  aircraft,  what  objects  the  pilot  will  encounter,  and  hov.'  the 
simulation  will  interact  with  the  pilot.  These  items  as  well  as  others,  which  are  transparent  to  the 
pilot,  are  specified  in  the  file  created  by  the  researcher.  The  scenario  creation  process  occurs  in 
three  steps:  1 )  use  a  text  editor  to  create  a  scenario  file  containing  the  definitions  of  the  elements 
of  the  scenario,  2)  pre-compile  the  scenario  file  into  a  combination  of  a  scenario  database  file  and 
auto-generated  C++  code  modules,  3)  recompile  the  simulation  program  to  include  the  new  code. 

Scenario  File 

The  creation  of  a  scenario  starts  with  the  creation  of  a  text  file  which  defines  the  elements  of  one 
or  more  scenarios.  The  overall  format  of  a  scenario  file  is  as  follows.  Note  that  the  spelling  and 
capitalization  of  underlined  keywords  in  the  file  is  important.  The  file  name  is  not  importan  :: 

[Start  of  File] 

Scenario:  scenario_number 


Scenarioi  scenario_number 


[End  of  File] 

In  the  above  diagram  the  keyword  Scenario:  is  followed  by  the  scenario  number.  Each  scenario 
in  the  file  must  have  a  unique  number.  The  number  of  the  first  scenario  can  optionally  be  le:'t  off 
and  the  generator  will  assign  it  the  number  "0". 
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Scenario  Elements 


Within  a  scenario  there  are  certain  objects  which  can  be  defined.  The  definitions  will  be  stored 
in  a  database  which  will  be  read  into  the  simulation  at  initialization  and  converted  into  actual 
simulation  entities.  The  order  in  which  objects  are  defined  within  the  scenario  is  unimportant 
except  for  triggers  which  must  follow  all  other  definitions.  The  following  is  a  list  of  definable 
objects  within  the  file: 

A.  Misc.  Objects 

1.  Comments 

2.  File  names 

3.  User  points 

4.  Way  points 

5.  SAM  sites 

6.  AAA  sites 

7.  Other  aircraft 

8.  User  defined  modes 

9.  Number  of  bombs  carried  by  the  pilot 

B.  Triggers 
Object  Syntax 

The  following  is  a  definition  of  each  object  and  the  syntax  for  specifying  it  in  the  file: 

A.  Misc.  Objects 

1 .  Comments  -  Comments  can  be  inserted  anywhere  in  the  file  for  explanatory  or 
documentation  purposes.  To  insert  a  comment  a  "//"  anywhere  in  the  line  will  cause 
it  to  be  ignored  by  the  generator. 

2.  File  names  -  One  or  more  file  names  can  be  declared  for  use  by  the  simulation  for 
both  reading  and  writing  data.  The  declaration  of  file  names  is  as  follows: 

Files: 

file_namel_tag ;  file_namel 
file_name2_tag  i  file_name2 
file_name3_tag  i  file_name3 


The  file  name  tag  is  a  reference  name  comprised  of  letters  and  numbers  up  to  15 
characters  in  length.  Examples  would  be:  FI,  TrimFile,  etc..  The  tag  is  the  name 
which  will  be  used  in  the  rest  of  the  scenario.  Following  the  colon  ( : )  is  the  tile 
name  as  it  appears  to  DOS.  Use  "W"  instead  of  "\"  to  specify  sub-directories  (i.e., 
C:\\a\\data\\trim  1  .dat). 
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3.  User  points  -  User  points  define  points  in  the  simulation  "world"  which  the  user 
wishes  to  refer  to.  A  good  example  of  their  use  is  in  specifying  the  location  of  the 
ILS  beacon  of  the  runway.  Following  is  the  syntax  for  user  points: 

user_pointl_tag  i  north_value  (nmi)  east_value  (nmi)  altitude  (ft) 
user_point2_tag  ;  north_value  (nmi)  east_value  (nmi)  altitude  (ft) 
user_point3_tag  i  north_value  (nmi)  east_value  (nmi)  altitude  (ft) 


Each  point  has  a  tag  name  followed  by  the  coordinates  of  the  point.  The  altitude 
coordinate,  if  left  off,  defaults  to  0. 

4.  Way  points  -  Way  points  define  the  points  along  the  way  of  the  pilot's  mission.  Way 
points  will  be  displayed  by  the  navigation  system  upon  activation.  Following  is  the 
syntax  for  way  points: 

Way_Points: 

way_pointl_tag  :  north_value  (nmi)  east_value  (nmi)  altitude  (ft) 
way_point2_tag  i  north_value  (nmi)  east_value  (nmi)  altitude  (ft) 
way_point3_tag  i  north_value  (nmi)  east_value  (nmi)  altitude  (ft) 


Each  point  has  a  tag  name  followed  by  the  coordinates  of  the  point.  The  altitude 
coordinate,  if  left  off,  defaults  to  0. 

5.  SAM  sites  -  SAM  sites  define  a  SAM  threat  location.  The  SAM  site  contains  a  radar 
for  tracking  the  pilot  if  the  pilot  is  in  range.  As  usual,  the  definition  includes  a  tag 
name  followed  by  the  coordinates  of  the  site.  Note  that  no  altitude  is  given  due  to 
the  assumption  that  SAM  sites  are  always  on  the  ground. 

SAM  Sites: 

sam_sitel_tag  i  north_value  (nmi)  east_value  (nmi) 
sam_site2_tag  :  north_value  (nmi)  east_value  (nmi) 
sam_site3_tag  ;  north_value  (nmi)  east_value  (nmi) 


6.  AAA  sites  -  AAA  sites  define  a  AAA  threat  location.  The  AAA  site  contains  a  radar 
for  tracking  the  pilot  if  the  pilot  is  in  range.  As  usual,  the  definition  includes  a  tag 
name  followed  by  the  coordinates  of  the  site.  Note  that  no  altitude  is  given  due  ro 
the  assumption  that  AAA  sites  are  always  on  the  ground. 

AAA  Sites: 

aaa_sitel_tag  :  north_vaIue  (nmi)  east_value  (nmi) 
aaa_site2_tag  1  north_value  (nmi)  east_value  (nmi) 
aaa_site3_tag  i  north  value  (nmi)  east  value  (nmi) 
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7.  Aircraft  -  Aircraft  other  than  the  pilot's  aircraft  can  be  specified.  Aircraft  have  a 
radar  system  like  the  SAM  and  AAA  sites.  Aircraft  are  defined  with  an  initial 
position  and  velocity.  Aircraft  fly  in  straight  lines  at  constant  velocity  but  their 
information  can  be  updated  by  the  simulation  to  simulate  intelligent  behavior. 

Aircraft: 

acl_tagi 

Position:  north_value  (nmi)  east_value  (nmi)  altitude  (ft) 

Velocity:  vel_north  (ft/min)  vel_east  (ft/min)  vel_up  (ft/min) 

ac2_tag  : 

Position:  north_value  (nmi)  east_value  (nmi)  altitude  (ft) 

V  elocity:  vel_north  (ft/min)  vel_east  (ft/min)  vel_up  (ft/min) 

ac3_tag : 

Position:  north_value  (nmi)  east_value  (nmi)  altitude  (ft) 

Velocity:  vel_north  (ft/min)  vel_east  (ft/min)  vel_up  (ft/min) 


8.  Modes  -  User  defined  modes  are  useful  when  certain  actions  are  to  be  limited  to 
certain  conditions  specified  by  the  researcher.  As  an  example,  use  of  the  trigger 
button  in  a  minimal  information  type  of  SA  scenario  can  be  enabled  only  during  the 
declared  mode  MfNIMAL  and  disabled  during  all  other  modes.  Following  is  the 
syntax  for  user  modes: 

Modes:  mode_namel,  mode_name2,  mode_name3,  etc. 

9.  Bombs  -  The  number  of  bombs  which  the  pilot  is  allowed  to  carry  is  specified  using 
the  following  syntax: 

Bombs: 

Number:  number 

B.  Triggers  -  Triggers  are  the  mechanisms  by  which  the  simulation  interacts  with  the  pilot  in 
ways  specified  by  the  researcher.  Triggers  respond  to  conditions  in  the  simulation  and 
execute  preprogrammed  operations.  For  a  discussion  of  triggers  plus  a  list  of  their 
components  see  the  document  "SAFTE  Theory  of  Operation".  Following  is  the  syntax  for 
declaring  triggers.  Note  that  triggers  must  follow  all  other  object  definitions  in  the  scenario. 
This  is  in  order  to  have  reference  to  all  tag  names  they  might  need: 

Triggers: 

triggerjag : 

Trigger  type: 

(GLOBAL,  LOCAL,  MODAL) 

Count:  (ALWAYS,  number) 

Enabled:  (YES,  NO) 

Conditions:  {...} 

Initial  Positive  Actions:  (...) 
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Positive  Actions:  {...} 
Initial  Negative  Actions: 
Negative  Actions:  { . . . } 


Each  trigger  has  three  parameter  fields  and  five  code  segments. 

The  three  types  of  triggers  are: 

GLOBAL  -  the  trigger  is  independent  of  any  special  condition  for  its 
evaluation  and  execution. 

LOCAL  -  the  trigger's  evaluation  and  execution  are  dependent  on  where 
the  pilot  is.  A  local  trigger  is  active  within  a  given  radius  from  a 
specified  point.  When  using  a  LOCAL  type  the  following  additional 
syntax  is  needed  (altitude  is  optional  and  defaults  to  0): 

Trigger  type: 

LOCAL: 

Center:  north_value  (nmi)  east_value  (nmi)  altitude  (ft) 
Radius:  radius  (ft) 

MODAL  -  the  trigger's  evaluation  and  execution  is  dependent  on  the 
current  mode  of  the  simulation.  See  user  defined  modes  above.  Syntax 
for  MODAL  is: 

Trigger  type: 

MODALi  mode_name 

The  Count  parameter  allows  the  trigger's  operation  to  be  limited  to  N  activations, 
or  have  no  limitations  (keyword  ALWAYS). 

The  Enabled  field  allows  the  trigger  to  be  enabled  or  disabled  at  the  beginning  of 
the  scenario.  Disabled  triggers  can  be  enabled  by  other  triggers  or  other 
mechanisms. 

The  five  code  segments  are  designed  to  contain  C++  code  using  the  object  definitions  within  the 
simulation  source  code.  Each  segment  can  define  an  arbitrary  sequence  of  instructions  but  must 
constitute  a  valid  C  function.  The  Conditions  segment  has  the  additional  restriction  that  it  must 
set  the  value  of  the  processor  status  flag  before  it  terminates.  This  allows  the  trigger  to  correctly 
evaluate  the  current  conditions.  The  actual  code  goes  between  the  {}s  in  the  above  syntax 
description. 

Creating  the  Scenario 

Once  the  scenario  file  has  been  created  the  next  step  is  to  pre-compile  the  file  into  an  objects 
database  and  a  set  of  auto-generated  code  modules.  The  steps  to  follow  are: 

a.  Have  the  scenario  file  in  the  SCENS  sub-directory  of  the  A  directory  on  the 
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simulation  machine. 

b.  From  the  A  directory  execute  the  batch  file  CS.BAT  which  will  clear  out  any  old 
scenario  data. 

c.  Type  "SIM  1  SCENS\FILENAME"  and  <enter>.  This  will  create  the  object 
database  and  the  auto-code  modules.  FILENAME  is  the  name  of  the  scenario  file  of 
interest. 

d.  Execute  the  batch  file  MAKW.BAT.  This  will  recompile  the  simulation  and  include 
the  new  code. 

The  SIM  program  and  the  compiler  will  display  warning  or  error  messages  if  any  syntactical 
errors  are  found.  Refer  to  the  set  of  .TXT  file  located  in  the  SCENS  directory  for  examples  of 
many  of  the  preceding  syntax. 

Terrain  Generator 

This  tool  was  designed  to  provide  a  graphical  interface  for  constructing  the  terrain  of  the  "world" 
in  which  the  pilot  flies.  In  addition  to  creating  terrain  the  terrain  generator  allows  the  following 
operations:  1)  importing  of  3D  objects  into  the  database,  2)  auto-shading,  and  3)  support  for 
day/night  and  weather  conditions. 

At  this  point  it  should  be  noted  that  many  of  the  terrain  generator  features  have  been  created 
though  a  process  of  experimentation.  Because  of  this  several  functions  of  the  generator  only 
work  partially  or  not  at  all.  The  following  procedures  guide  will  help  direct  a  potential  user 
through  this  disjoint  environment. 

Viewing  Terrain 

One  of  the  most  useful  (and  functional)  feature  of  the  generator  is  v  iewing  the  existing  terrain. 

For  the  researcher  who  wishes  to  modify  existing  scenarios  or  create  new  ones,  the  terrain 
viewing  feature  allows  the  researcher  to  pinpoint  locations  on  the  map  which  can  subsequently 
be  entered  into  the  scenario  file  as  user  points,  way  points,  or  SAM  site  or  AAA  site  locations. 

The  first  step  in  viewing  is  to  read  the  terrain  database  into  memory.  Type  "VIS"  and  <enter>  to 
make  sure  that  the  current  directory  is  the  C:\VIS  directory.  Make  sure  that  the  latest  version  of 
the  terrain  database  (name:  TERRAfN.DAT)  resides  in  this  directory.  Start  the  viewer  by  typing 
"EX"  and  <enter>.  When  the  program  starts  a  set  of  rectangular  areas  will  be  visible  along  with 
the  mouse  cursor  in  the  upper  left  portion  of  the  screen.  Move  the  cursor  over  the  R  symbol  at 
the  lower  right  of  the  screen  and  single  click  with  the  mouse.  In  a  few  seconds  the  middle  area  of 
the  screen  will  fill  with  the  images  of  the  terrain  and  object  elements. 

Moving  the  cursor  over  the  picture  will  cause  numbers  in  the  lower  left  portion  of  the  screen  to 
change.  These  numbers  are  the  East  and  North  coordinates  of  the  areas  over  which  the  mouse 
lies. 

Move  the  cursor  over  the  yellow  down  arrow  at  the  lower  right  of  the  screen  and  single  click 
again.  Moving  the  cursor  over  the  terrain  map  now  results  in  a  box  being  drawn  over  the  same 
quadrant  of  the  map  that  the  cursor  lies.  Clicking  on  the  map  causes  the  generator  to  zoom  into 
the  chosen  quadrant.  At  the  lower  left  of  the  screen  to  the  left  of  the  mouse  coordinates  is  a  field 
named  "level".  This  value  indicates  how  many  levels  down  the  viewer  currently  is  in  the  map. 
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Each  level  is  exactly  one  half  of  the  dimensions  of  the  upper  level.  Clicking  on  the  yellow  up- 
arrow  reverses  the  view.  There  are  currently  12  levels  supported  for  viewing. 

Creating  and  Deleting  Objects 

Objects  in  the  terrain  editor  are  created  one  facet  (or  face)  at  a  time  and  laid  down  like  sheets  one 
on  top  of  another.  This  means  that  objects  like  mountains  can  be  constructed  and  placed  on  top 
of  plains,  but  a  topologically  more  complex  object  like  a  ball  cannot  be.  An  object  can  be 
constructed  as  one  or  more  joined  facets  or  as  a  more  complex  arrangement  of  groups  of  joireci 
and  disjoint  facets.  Note:  only  convex  polygonal  objects  can  be  specified.  Groups  of  objects 
can  be  formed  to  create  more  complex  shapes. 

When  creating  objects  the  color  of  the  object  and  the  3D  position  of  each  vertex  are  specified. 

To  begin  constructing  a  simple  object  (i.e.,  one  set  of  joined  facets)  click  with  the  left  mouse 
button  on  the  "C"  symbol  at  the  lower  right  portion  of  the  screen.  A  window  will  appear 
showing  a  selection  of  colors  to  choose  from.  Click  on  a  color  from  the  palette  of  colors  shewn. 
If  the  selected  color  needs  to  be  modified,  hold  down  the  left  button  with  the  cursor  over  the 
RGB  bars  and  slide  each  bar  up  or  down  to  change  the  components  of  the  color  to  create  the 
desired  shade.  When  finished,  click  the  left  mouse  button  on  the  red  square  at  the  lower  left  of 
the  window  to  close  it. 


Having  selected  the  object's  color  move  the  cursor  to  the  point  on  the  map  where  the  first  vertex 
of  the  object  will  be.  Click  once  over  the  position  with  the  left  button  and  another  window  vrtll 
appear  allowing  the  altitude  of  the  point  to  be  specified.  Move  the  altitude  bar  up  or  down  by 
dragging  the  cursor  up  or  down  with  the  left  button  pressed  over  the  bar  area.  For  fine 
adjustments  use  the  blue  up  and  down  arrows. 

Once  the  altitude  of  the  first  point  is  chosen  three  options  are  available.  If  the  altitude  of  each 
vertex  needs  to  be  specified  independently  click  on  the  red  box  in  the  altitude  window.  Each 
time  a  new  vertex  point  is  picked  the  altitude  window  will  appear  allowing  the  altitude  of  the 
point  to  be  selected.  If  the  altitude  of  the  entire  object  is  to  be  the  same  as  the  first  point  click  on 
the  green  button  after  selecting  the  first  point's  altitude.  From  this  point  until  the  object 
definition  is  finished  the  altitude  box  will  not  appear  again  and  the  altitude  of  each  new  point 
will  be  automatically  assigned  the  altitude  of  the  first  point.  This  option  is  useful  for  quickly 
creating  flat  pieces  of  terrain. 

If  the  position  of  a  vertex  is  desired  to  coincide  with  the  position  of  the  vertex  of  another  object 
then  the  yellow  key  is  used.  To  use  this  option  the  cursor  needs  to  be  placed  over  the  other 
vertex  before  the  left  button  is  pressed.  When  the  yellow  key  is  selected  the  generator  will  recall 
the  coordinates  of  the  selected  vertex  and  assign  these  values  to  the  new  vertex.  This  option  is 
useful  for  aligning  different  objects  to  create  a  mosaic. 

To  finish  the  object's  definition  press  the  right  button  after  the  last  vertex  has  been  entered.  The 
generator  will  automatically  close  the  polygon  so  defining  an  extra  point  is  not  necessary. 

To  combine  groups  of  facets  into  more  complex  objects  precede  the  above  steps  by  clicking  the 
"O"  symbol  at  the  lower  right  of  the  screen.  Now  an  arbitrary  number  of  groups  of  facets  can  be 
created  with  the  above  steps  and  assigned  to  one  object.  Close  the  new  object's  definition  b) 
clicking  again  on  the  "O"  symbol. 
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To  delete  an  object  click  on  the  "D"  symbol  at  the  lower  right  comer  of  the  screen  then  click  on 
the  object  to  be  deleted. 

Copying  and  Moving  Objects 

To  move  an  object  click  on  the  "M"  symbol  at  the  lower  right  of  the  screen,  click  on  the  object  to 
be  moved,  then  click  on  the  map  where  the  object  is  to  be  placed.  The  screen  will  be  redrawn 
with  the  object  in  the  new  position. 

To  copy  an  object  follow  the  above  steps  but  select  the  "cp"  symbol  instead.  The  screen  will  be 
redrawn  showing  both  the  old  and  the  new  objects. 

Scaling  and  Rotation 

To  scale  an  object's  size  up  or  down  press  the  (minus)  or  the  "+"  (plus)  key  from  the  keyboard 
then  click  on  the  object.  The  scaling  increment  is  currently  set  at  5%  of  the  current  dimensions 
of  the  object. 

To  rotate  left  or  right  press  the  "1"  or  "r"  key  on  the  keyboard  and  click  on  the  object.  The 
increment  of  rotation  is  2°  at  present. 

Editing  an  Object 

To  edit  the  color  or  vertices  of  an  object  the  "E"  symbol  is  first  selected.  To  change  the  position 
and/or  altitude  of  a  vertex  position  the  cursor  over  the  vertex  and  click  on  it  with  the  left  button. 
Move  the  cursor  to  the  new  position  and  click  again.  The  altitude  window  will  appear  allowing 
the  altitude  of  the  point  to  be  modified.  Once  the  altitude  is  selected  the  screen  will  be  redrawn 
showing  the  modified  object. 

To  change  the  color  of  an  object  click  within  the  bounds  of  the  object  away  from  any  vertices. 
The  color  window  will  appear  allowing  a  new  color  to  be  selected  or  the  currently  selected  color 

to  be  modified. 

Importing  3D  Objects 

Additional  terrain  or  full  3D  objects  can  be  imported  into  the  database  for  use  in  the  simulation. 
This  feature  makes  use  of  a  separate  program  called  CONVERT.EXE.  The  steps  to  do  this  are: 

1 .  Create  an  object  in  a  text  editor  according  to  the  format  shown  in  diagram  1 . 

2.  Using  a  text  editor  strip  out  all  information  in  the  TERRAfN.DAT  except  that  which 
defines  the  set  of  base  colors  (located  at  the  beginning  of  the  file)  and  that  which 
defines  the  objects  (located  at  the  end  of  the  file). 

3.  Insert  the  new  object's  definition  in  the  file  and  save  the  information  to  a  new  file. 

4'  Execute  "CONVERT  FILE_IN  FILE_OUT"  where  FILEJN  is  the  file  just  created 

and  F1LE_0UT  is  the  new  terrain  file. 

5.  Replace  TERRAIN.DAT  with  FILE_OUT. 
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Shading 

To  auto-shade  the  objects  in  the  database  for  day/night  and  weather  conditions  (clear/cloudy)  hit 
the  s  key  on  the  keyboard.  The  program  will  indicate  when  shading  is  complete. 

Saving  the  Data 

To  update  the  database  click  on  the  "S"  symbol  at  the  lower  right  comer  of  the  screen.  The 
database  is  now  ready  to  be  copied  to  the  DATA  sub-directory  of  the  graphics  machine  and 
executed. 
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APPENDIX  4 


INSTRUCTIONS  FOR  THE  SAFTE  SIMULATOR  CONTROLS 


Keyboard  Switches 

1  -  landing  gear  toggle 
w  -  toggle  waypoints 
t  -  toggle  between  chaff  and  flares 
r  -  reset  max  G  counter  and  the  flight  director  system 

keyboard  switches  must  be  in  lower  case  to  work  correctly 

Joystick  Switches 

Red  button  under  thumb  at  top  -  weapons  release  button 

Small  black  hat  button  under  weapons  release  button  -  up/down  for  yes/no  answers  to  SA 
questions 

Red  trigger  button  on  front  -  minimal  information  release  button 
Red  paddle  button  on  front  bottom  -  chaff/flare  release 
Throttle  Switches 

Top  white  three  way  switch  -  mode  switch: 

Near  -  Air-to-Ground 
Middle -NAV 
Far  -  Air-to-Air 

Two  position  white  switch  under  mode  switch  -  speed  brake: 

Near  -  incremental  extension  (hold  for  maximum) 

Far  -  one  shot  automatic  retraction 

Cursor  under  thumb  -  Air-to-Air  radar  target  selector  cursor 
Cursor  button  -  select  target 
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APPENDIX  5 


SAFTE  USER^S  MANUAL 


The  SAFTE  system  is  designed  to  provide  researchers  with  an  open-ended  tool  for  collecting 
data  on  a  range  of  situational  awareness  (SA)  measures  within  the  context  of  a  real  life  task  - 
flying  an  airplane.  The  SAFTE  system,  as  delivered,  comes  with  a  medium-to-high  fidelity 
model  of  an  F-1 6  aircraft,  a  simulated  world  database  to  fly  around  in,  and  a  set  of  83  prepared 
scenarios  with  embedded  SA  measures.  The  system  also  includes  tools  capable  of  both  editing 
the  current  graphics  database  and  scenarios,  and  creating  new  ones. 

This  Appendix  is  written  to  assist  users  in  setting  up  the  SAFTE  system,  executing  existing 
scenarios  and  converting  collected  data  into  a  useful  format  for  analysis.  For  a  detailed 
description  of  the  operating  principles  of  the  SAFTE  system,  or  of  the  scenario  and  terrain 
editing  tools,  refer  to  Appendices  I  through  4  of  this  report. 

Overview 

The  SAFTE  system  is  physically  comprised  of  two  Pentium  computers  connected  via  their 
parallel  ports  with  a  custom  supplied  cable.  In  this  system,  one  machine  is  the  simulation 
computer  and  the  other  is  the  graphics  computer.  Inputs  from  the  controls  run  to  the  simulation 
machine  and  information  is  transmitted  to  the  graphics  machine  through  the  parallel  interface. 

In  addition,  the  SAFTE  system  comes  equipped  with  a  full  set  of  control  input  devices:  a  flight 
stick  (ThrustMaster  FLCS),  a  throttle  (ThrustMaster  TQS)  and  rudders  (ThrustMaster  RCS). 
These  devices  are  fitted  with  HOTAS  (Hands  On  Throttle  and  Stick)  controls  which  allow 
commands  to  be  executed  without  having  to  remove  ones  hands  from  the  keyboard.  It  is 
recommended  that  the  manufacturer's  documentation  be  reviewed  prior  to  use  in  order  to  gain  an 
understanding  of  the  placement  and  operation  of  each  of  the  controls  used  by  the  system.  Note 
that  SAFTE  only  uses  a  portion  of  the  switches  available  on  the  controls,  so  a  complete 
familiarization  is  not  needed. 

The  execution  of  scenarios  can  be  done  in  one  of  two  ways:  individually  or  as  a  programmed 
sequence.  Auto-sequencing  (discussed  later)  allows  the  researcher  to  step  a  subject  through  a 
custom  set  of  scenarios  without  the  subject  having  to  remember  each  scenario.  Whichever  way 
is  chosen,  a  uniquely  named  data  file  is  generated  for  each  scenario. 

Setting  Up  the  Equipment 

The  physical  layout  of  the  system  consists  of  two  computers.  The  fastest  machine  hosts  the 
graphics  software  and  the  slower  machine  hosts  the  simulation  software,  as  well  as  being 
equipped  with  the  ThrustMaster  game  card.  (See  the  "SAFTE  User’s  Manual”  —  Appendix  5,  for 
detailed  specifications  concerning  hardware  requirements.) 

Begin  configuration  of  the  system  by  setting  the  computers  closely  together  and  parallel  to  each 
other.  Position  the  monitors  near  the  computers,  leaving  enough  space  for  the  flight  stick, 
throttle,  and  the  Inteva  keyboard.  Remember  to  position  the  equipment  so  that  the  graphics 
monitor,  the  keyboard,  and  the  controls  are  in  a  comfortable  arrangement.  Set  the  rudders  on  the 
floor  directly  underneath  the  other  controls. 
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After  physically  positioning  all  the  equipment,  start  assembling  the  system  by  hooking  power 
cables  to  the  computers  and  monitors  and  connecting  the  two  computers  together  by  attaching  the 
supplied  parallel  cable  to  each  computer's  parallel  port.  Connect  the  Dell's  keyboard  to  the  top 
P2  connector  on  the  back  of  the  machine  and  connect  the  standard  PC  connector  from  the  flight 
stick  to  the  keyboard  input  of  the  Inteva.  The  Inteva's  keyboard  will  be  the  primary  keyboard 
input  so  the  Dell's  keyboard  may  be  placed  to  one  side. 

Next,  connect  the  "D"  connector  of  the  throttle's  cable  to  the  right  hand  side  game  port  of  the 
Inteva  (as  seen  from  the  back),  then  use  the  correct  gender  connector  on  the  rudder  cable  to 
connect  the  rudders  to  the  back  of  the  throttle.  The  "D"  connector  cable  coming  from  the  flight 
stick  can  now  be  connected  to  the  other  end  of  the  rudder  cable.  Connect  the  Inteva's  keyboard 
to  the  keyboard  input  of  the  flight  stick.  At  this  point  all  the  hardware  has  been  connected  and 
the  system  can  now  be  powered  up.  If  computers  different  than  those  supplied  under  this  effort 
are  being  used,  refer  to  the  system  software  requirements  in  the  "SAFTE  Theory  of  Operation" 
section  of  this  report  to  verify  that  all  required  software  is  present. 

Turn  on  each  computer  and  verily  that  both  machines  correctly  power  up.  If  the  systems  are 
configured  properly  the  graphics  machine  should  begin  executing  the  graphics  program 
automatically  after  power  up.  If  not,  refer  to  the  'Troubleshooting'  section  of  this  Appendix  for 
debugging  assistance. 

At  this  point,  if  machines  other  than  the  supplied  systems  are  being  used,  an  additional  step  will 
need  to  be  performed.  Reboot  each  machine  in  turn  and  enter  the  setup  menu  at  the  start  of  boot. 
Locate  the  portion  of  the  menu  specifying  which  parallel  port  protocol  should  be  used  and  verify 
that  it  is  set  to  allow  standard  bi-directional  transfer.  The  terminology  of  different  computer 
manufacturers  can  be  different,  so  some  experimentation  may  be  necessary.  Again,  see  the 
section  on  troubleshooting  if  problems  persist. 

Verify  that  both  machines  are  up  and  running  and  the  simulation  computer  is  at  the  DOS  CP- 
prompt.  If  this  is  the  first  time  this  equipment  has  been  set  up,  then  on  the  simulation  computer 
type  "TMS"  and  <enter>.  This  will  invoke  the  ThrustMaster  utilities  program,  which  will  be 
used  to  program  the  ThrustMaster  controls.  Follow  the  instructions  for  downloading  files, 
specifying  the  file  SAFTE.B50  under  the  PROGRAMS  sub-directory.  Refer  to  the  ThrustMaster 
documentation  for  information  on  the  TMS  program  and  its  use. 

After  completing  the  controls  programming,  type  "A"  and  <enter>.  This  will  transfer  operations 
to  the  simulation  directory  where  the  simulation  will  be  run  and  data  files  generated. 

Before  running  the  simulation,  the  stick,  throttle  and  rudders  must  be  calibrated.  To  do  this,  two 
procedures  must  be  followed:  First,  before  the  calibration  routine  is  run,  the  SI  button  on  the 
upper  right  of  the  flight  stick  MUST  be  pressed.  This  is  to  enable  the  programming  of  the 
controls.  Second,  run  the  calibration  program  by  typing  "CAL"  and  <enter>.  The  SAFTE 
calibration  program  will  come  up  with  a  graphical  and  numerical  presentation  of  the  state  of  the 
controls.  Using  the  paddle  button  on  the  stick's  bottom  front,  and  the  trigger  button  on  the  stick's 
front,  step  through  the  calibration  of  each  control,  remembering  to  fully  deflect  the  roll,  pitch, 
and  rudders,  and  accurately  position  the  throttle  at  each  detent  point.  To  end  the  calibration,  ty  pe 
"X"  and  return. 

Running  The  Scenarios 
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Once  the  controls  have  been  programmed  and  calibrated,  the  system  is  ready  to  be  used  for  data 
collection.  If  only  a  specific  scenario  is  desired,  then  simply  type  ”S’’  and  <retum>. 

When  the  program  begins,  a  message  asking  for  the  subject's  ID  number  will  appear.  When  the 
subject’s  number  is  entered,  the  program  then  asks  for  the  number  of  the  scenario  to  run.  Afi;er 
the  scenario  number  is  entered,  a  message  appears  on  the  graphics  screen  with  details  of  the 
scenario.  The  scenario  will  begin  after  a  key  is  pressed.  At  the  end  of  the  scenario,  the  program 
will  exit,  having  created  a  data  file  in  the  simulation  directory  (C:\A). 

If  an  automated  sequence  of  scenarios  is  desired  the  following  procedure  is  used: 

0.  Insert  a  floppy  disk  into  the  A:  drive.  This  disk  should  contain  the  auto-sequencing  file. 
From  the  simulation  directory  type  ’’MAKEDISK"  and  <return>.  The  MAKEDISK  program 
will  ask  for  the  subject’s  number,  then  the  number  of  scenarios  to  execute,  then  finally  the 
number  of  each  scenario  in  order. 

1 .  Once  the  MAKEDISK  program  has  created  the  auto-sequencing  file  (called  SAFTE.DAT) 
on  the  disk  the  subject  is  ready  to  execute  the  sequence  of  scenarios. 

2.  To  start  the  simulation,  insert  the  disk  into  the  A:  drive  and  type  ”M”  from  the  simulation 
directory  and  <enter>.  The  program  will  begin  by  asking  for  the  subject's  ID  number.  If  the 
number  matches  that  on  the  disk  file,  the  next  action  of  the  program  will  be  to  display  a 
message  on  the  graphics  screen  with  the  details  of  the  first  scenario  to  be  run.  From  this 
point  on  the  program  behaves  the  same  as  the  single  scenario  case.  When  the  scenario 
finishes  running,  the  simulation  will  return  to  the  subject  ID  query.  The  subject  can  con:inue 
and  execute  the  next  scenario  or  quit  the  program  entirely  and  resume  at  a  later  date. 

A  few  points  should  be  mentioned  about  using  the  auto-sequencing  feature. 

1 .  Each  subject's  ID  must  be  registered  with  the  program  beforehand  by  including  it  in  the  file 
IDS. DAT  in  the  sub-directory  of  the  simulation  directory  (C:\A\SAFTEDAT),  This  is  a  text 
file  with  each  ID  number  listed  in  order. 

2.  If  the  subject's  ID  has  been  registered  on  each  SAFTE  system  in  use,  the  subject  is  free  to  use 
any  system  he  or  she  chooses  by  simply  inserting  the  floppy  with  the  auto-sequencing  disk  in 
the  floppy  drive  of  the  selected  machine. 

3.  If  the  subject  aborts  out  of  a  scenario  by  hitting  the  ESC  key  the  program  will  allow  the 
scenario  to  be  run  one  more  time.  After  this,  the  program  will  advance  to  the  next  scenario 
regardless  of  the  outcome  of  the  repeated  scenario. 

Control  and  Keyboard  Inputs 

Input  to  the  simulation  comes  from  two  main  sources:  the  controls  and  the  keyboard.  The  control 
inputs  can  be  further  divided  into  analog  inputs  (roll,  pitch,  rudder,  throttle)  and  discrete  inputs 
(trigger,  weapons  release  button,  etc.).  Following  is  a  list  of  the  discrete  controls  used.  After  the 
controls  list  is  a  list  of  the  valid  keyboard  inputs.  Refer  to  the  ThrustMaster  documentation  for 
locations  and  further  descriptions  of  each  control: 
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Control  Inputs 


Weapons  Release  Button  -  Located  at  the  top  back  of  the  flight  stick.  Press  to  launch 
bombs  or  missiles. 

Trigger  -  Located  at  the  front  top  of  the  flight  stick.  Hold  in  to  clear  the  screen  during 
minimum  information  mode. 

Slider  Switch  -  Black  four  position  switch  located  just  under  the  weapons  release  button 
on  the  flight  stick.  Answers  SA  probe  questions:  Up  is  yes,  down  is  no. 

Paddle  Switch  -  Red  paddle  switch  located  at  the  bottom  front  of  the  flight  stick.  Used  to 
dispense  chaff  or  flares  (two  at  a  time). 

S I  Button  -  Red  button  located  at  the  top  right  of  the  flight  stick.  This  button  enables  the 
control  programming.  It  MUST  be  pressed  at  the  beginning  of  each  session  on 
the  system  after  power  up. 

Mode  Switch  -  Three  way  white  switch  located  on  top  of  the  throttle.  When  pulled 

toward  the  pilot  it  enables  CCIP  (air-to-ground)  mode.  NAV  (navigation)  mode 
is  in  the  middle  and  AAM  (air-to-air  mode)  is  the  position  away  from  the  pilot. 

Speed  Brake  -  Two  position  white  switch  located  below  the  Mode  switch  and  toward  the 
pilot  on  the  throttle.  This  switch,  when  pulled  toward  the  pilot,  deploys  the 
speed  brake,  and  when  pushed  away,  retracts  the  speed  brake.  Deployment  :  s 
proportional  to  the  amount  of  time  the  switch  is  pulled  whereas  retraction  is 
automatically  full  and  complete. 

Track  Ball  /Mouse  Button-  Track  ball/cursor  knob  located  under  the  thumb  on  the 

throttle.  This  control  is  for  the  air-to-air  radar  cursor  used  in  designating  targets. 
Placing  the  return  inside  the  cursor  bounds  and  clicking  the  white  '’mouse  key” 
switch  located  near  the  cursor  control  will  designate  the  selected  target. 

Keyboard  Inputs 

c  -  Clears  the  graphics  screen  of  any  messages  and  returns  to  normal  OTW  (Out-The- 
Window)  processing 

1  -  Toggles  landing  gears 

t  -  Toggles  countermeasures  between  chaff  and  flares 

w  -  Selects  the  next  way  point 

Esc  -  Aborts  out  of  a  scenario  or  quits  the  program  and  returns  to  DOS 

Note:  The  keyboard  inputs  MUST  be  in  lower  case  to  function  properly. 

Data  Analysis 

Whether  scenarios  are  run  individually  or  as  sets,  the  result  is  the  creation  of  a  data  file  for  each 
scenario  containing  a  history  of  the  pilot's  activity  and  the  state  of  the  aircraft.  Each  data  file  has 
a  unique  name  that  identifies,  among  other  things,  the  subject  ID,  the  scenario  number  and  the 
reason  for  terminating  the  scenario  (crash,  time  out,  landing,  etc.).  ***Diagram  2  of  the  appendix 
contains  a  definition  of  each  portion  of  the  file  name. 

Note  that  when  auto-sequencing  scenarios,  the  file  names  will  all  have  unique  names  derived 
from  data  contained  in  the  SAFTE.DAT  file  on  disk.  If  scenarios  are  run  individually  there  is  a 
possibility  of  creating  data  files  with  identical  names.  The  researcher  is  cautioned  to  review  his 
or  her  procedures  being  used  before  commencing  data  collection  in  this  mode. 

The  contents  of  the  data  files  is  comprised  of  a  collection  of  different  data  types  having  different 
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recording  frequencies.  The  following  list  describes  the  types  of  data  and  their  rate  of  occurrence. 
Refer  to  the  previous  appendices  for  a  complete  description  of  these  data  records: 

Data  File  Data  Types 

Aircraft  Output  Vector  -  The  vector  of  information  detailing  the  aircraft's  position, 

velocity,  attitude  and  other  parameters  is  transmitted  to  the  data  file  once  pe- 
second. 

Control  Input  Vector  -  The  vector  of  information  detailing  the  values  of  the  roil,  pitch, 
throttle,  and  rudders  is  transmitted  to  the  data  file  five  (5)  times  per  second. 
Keyboard/Control  Character  Input  Record  -  A  character  from  the  keyboard  or  from  the 
discrete  controls  gets  logged  to  the  data  file  when  it  is  entered. 

SA  Record  -  Situational  Awareness  (SA)  record  of  an  activity  of  an  SA  nature.  The 
possible  activities  are  detailed  in  the  appendix.  SA  records  are  logged  to  the 
data  file  as  they  occur. 

The  following  information  is  for  researchers  who  are  performing  data  analysis  using  the  SAFTT 
data  format.  Each  data  record  is  time  stamped  with  the  time  at  which  the  data  record  was 
created.  The  starting  point  of  the  time  value  is  not  the  beginning  of  the  scenario,  but  rather  the 
start  of  the  program.  While  the  program  is  running,  there  is  a  "real"  time  counter  that  keeps  track 
of  hours,  minutes,  seconds  and  fractional  seconds.  This  is  the  total  elapsed  time  since  the 
simulation  started.  A  second  counter,  the  ’’simulation"  time  counter,  keeps  track  of  the  time  the 
scenario  has  actually  run.  It  does  not  count  the  “pauses”  that  occur  during  SA  probes.  Time 
stamping  data  with  "real"  time  values  allows  for  the  timing  of  events  which  occur  while  the 
simulation  is  halted,  such  as  when  waiting  for  the  pilot  to  respond  to  an  SA  probe  message. 

The  data  set  chosen  for  logging  purposes  is  not  a  complete  record  of  every  occurrence,  but  rather 
an  essential  set  of  data  from  which  other  values  can  be  inferred.  A  simple  but  important  example 
of  this  is  determining  the  start  and  stop  of  "simulation"  time.  There  exists  an  SA  record  which 
records  the  enabling  and  disabling  of  the  simulation  time  counter  ("1"  means  enabling,  "0"  means 
disabling).  From  the  activity  of  this  record  (called  SIM  TIME)  a  current  value  of  simulation  time 
can  be  constructed. 

Data  Extraction 

The  data  contained  in  the  data  file  is  in  a  binary  (non-text)  format.  To  convert  the  data  to  a  text 
format  suitable  for  reading  or  input  into  a  spreadsheet,  the  program  FILTER.EXE  is  provided. 

To  convert  a  file  use  <chdir>  to  change  to  the  simulation  directory,  then  type  "FILTER  <data  file 
name>  <output  file  name>"  and  <enter>.  For  full  length  scenarios  running  20  minutes  or  longer 
the  output  file  size  can  be  considerably  greater  than  one  (1)  megabyte  in  size.  Each  line  of  the 
output  file  is  in  the  format: 

[Time  Stamp]  [Data  Record  Type]  [Data] 

Be  aware  that  for  some  records  the  line  length  may  be  substantially  more  than  80  characters. 
Troubleshooting 

The  following  is  a  basic  troubleshooting  guide  for  common  problems: 

Problem:  Nothing  happens  on  the  graphics  side  when  the  simulation  is  run. 
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Solution:  Verify  that  the  parallel  port  cable  is  securely  connected  between  the  two 
machines.  If  this  does  not  solve  the  problem  the  parallel  port  protocol  specified  by  each 
computer's  setup  program  may  not  be  the  correct  one.  Some  e.xperimentation  may 
needed  to  find  the  correct  values. 

Problem:  The  ThrustMaster  controls  do  not  seem  to  work. 

Solution:  Make  sure  the  red  SI  button  on  the  flight  stick  has  been  pressed  since  power  up 
(but  see  below). 

Problem:  The  controls  will  not  work  even  after  pressing  the  S 1  button,  or  only  some 
controls  function  but  not  others. 

Solution:  If  the  SI  button  has  been  pressed,  quit  the  program  and  from  the  DOS 
command  line  try  operating  the  controls.  If  no  characters  appear  on  the  command  line 
then  those  controls  are  possibly  are  defective,  or  the  device  itself  is  defective. 

Problem:  "Config  file  failure”  or  "file  not  found"  messages  appear. 

Solution:  Verify  that  the  path  statement  is  correct.  The  easiest  way  to  do  this  is  to  make 
sure  the  AUTOEXEC.BAT  and  CONFIG.SYS  files  supplied  with  the  system  are  being 
used.  Reboot  the  computer  to  enable  any  changes  to  the  AUTOEXEC  or  CONFIG  files. 


Diagram  5.1 

Hardware  Layout  of  SAFTE  System 
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A  ABBBCCD.EEE 


A  -  Scenario  number  (0  -  99) 

B  -  Cumulative  number  of  runs  (I  -  999) 

C  -  Number  of  repeats  for  current  scenario  (0  -  99) 
D  -  Ending  scenario  cause: 

1  -  Computer  crash 

2  -  Successful  ending 

3  -  Time  out 

4  -  Crash  landing 

5  -  User  abort 

6  -  System  operator  abort 
E  -  Subject  number 


Listing  5.1 

SAFTE  Data  File  Name  Format 


Position  Name 

Description 

1 

AZN 

//Normal  accel.  in  gs 

2 

AYN 

//  Lateral  accel.  in  gs 

3 

AXN 

//  Long,  accel.  in  gs 

4 

OBAR 

//  Dynamic  pressure  in  psf 

5 

MACH 

//  Mach  number 

6 

VKT 

//  True  airspeed 

7 

AOA 

//  AOA  in  deg. 

8 

BETA 

//  Slide  Slip  in  deg. 

9 

PHID 

//  Pitch  in  deg. 

10 

THTAD 

//  Roll  in  deg. 

11 

PSID 

//  Heading  in  deg. 

12 

PD 

//Roll  rate  in  deg/sec 

13 

QD 

//  Pitch  rate  in  deg/sec 

14 

RD 

//  Yaw  rate  in  deg/sec 

15 

NORTH 

//  Position  North  in  ft. 

16 

EAST 

//  Position  East  in  ft. 

17 

ALT 

//  Position  Down  (Alt.)  in  ft. 

18 

QC 

19 

QCOPS 

20 

VNORTH 

//  Veloc.  North  in  ft/sec. 

21 

VEAST 

//  Veloc.  East  in  ft/sec. 

22 

VDOWN 

//  Veloc.  Down  (Alt.)  in  ft/sec. 

23 

HDOT 

//  Sink  rate  in  ft/min. 

24 

CPOW 

//  Commanded  power. 

25 

VCAS 

//  Calibrated  airspeed 

Listing  5.2 

Aircraft  Output  Vector 
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Position  Name 

Description 

1  ROLL 

//  “  1 .0  to  1 .0  full  deflection 

2  PITCH 

//-l.O  to  1.0  full  defiection 

3  RUDDER 

//  -  1 .0  to  1 .0  full  defiection 

4  THROTTLE 

//  0.0  to  1 .0  full  throttle  (AB) 

Note:  Actual  values  may  fail  outside  stated  ranges  due  to  calibration 
inaccuracies. 

Listing  5.3 

Controls  Input  Vector 


Button 

Code 

Character 

SI 

65 

Push  -  "A" 

66 

Release  -  "B" 

Trigger 

49 

Push-”1" 

48 

Release  -  "0" 

Slider  Up 

72 

Push  -  ”H" 

Slider  Down 

73 

Push-"!" 

Slider  Return 

76 

Return  -  "L" 

Weapons  Rel. 

45 

Press  - 

"Pinky"  btn. 

51 

Press  -  "3" 

"Pinky"  btn. 

50 

Release  -  "2" 

Paddle  btn. 

46 

Press  - "." 

CCIP  mode 

34 

Press  -  """ 

NAV  mode 

35 

Return  - 

ATA  mode 

36 

Press  - 

Speed  Brake  Ex. 

37 

Press  -  "%" 

38 

Return  - 

Speed  Brake  Rt, 

39 

Press  - 

ATA  Designate 

40 

Press  -  "(" 

ATA  Cursor,  R 

41 

Press  -  ")" 

ATA  Cursor,  L 

42 

Press  - 

ATA  Cursor,  D 

43 

Press  - 

ATA  Cursor,  U 

44 

Press  -  "," 

Landing  Gears 

108 

keyboard  "1" 

Counter  Meas. 

116 

keyboard  "t" 

Way  Points 

119 

keyboard  "w" 

Note:  This  is  an  abbreviated  listing.  Please  refer  to  the 
SAFTE.B50  file  for  complete  information. 

Listing  5.4 

Control  Character  Codes 
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Value 

Name 

Descriotion 

0 

SIM^MESSAGE 

Display  message 

1 

SFM  TGT_DESIG 

Designate  target 

2 

SIM  UNUSUAL  ATT 

Do  unusual  attitude 

3 

SIM  TRIGGER  ACTV 

Simulation  trigger  activation 

4 

SIM  TIME 

Start/stop  of  simulation  time 

5 

SIM  ATA  RADAR  TT A  TTA  of  designated  target  | 

6 

SIM  RWR  RADAR 

New  guy  RWR  indication 

7 

SIM_CUR_WP 

Selected  way  point  value 

Listing  5.5 

Recorded  SA  Activities 
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APPENDIX  6 


TRAINING  SCENARIOS  PROGRAMMED  IN  SAFTE 


Training  scenarios 

SUBJECT  INSTRUCTIONS  CHECKLIST  FOR  THE  SABER  PROJECT 
PRE-TRAINING 


Before  the  first  hour  of  training,  the  subject  will  have  been  given  copies  of  the  “GENERAL 
TRAINING  INSTRUCTIONS  FOR  AIR-TO-GROUND  ATTACK  MISSIONS”  and  the 
“GENERAL  TRAINING  INSTRUCTIONS  FOR  COMBAT  AIR  PATRIOL  (CAP)  MISSION: 
It  is  expected  that  they  will  have  read  through  these  prior  to  reporting  for  the  first  hour  of 
training. 


HOUR  1  FOR  EACH  SUBJECT  (TOTAL  TIME  EST.  =  48  minutesJ 

INTRODUCTION  TO  THE  PROJECT  (10  minutes) 

Give  a  general  introduction  to  the  goals  and  background  of  the  project 
AF  SBIR  -  expected  to  be  used  in  AF  training  and  selection 
Constructed  desk-top  flight  simulator  with  embedded  SA  measures 
Highest  quality  aerodynamics  available  on  PC 

Current  status  -  still  a  “prototype”  -  we  expect  some  systems  problems 

SABER  project  -  determine  basic  learning  curves  for  different  experience  levels  on 

different  maneuvers 

Performance  expected  -  DO  YOUR  BEST  -  not  expected  to  be  “fighter  pilots”.  Some 
missions  may  prove  too  difficult  -  this  is  OK. 

No  scoring  will  be  reported  to  UND  -  absolute  privacy  of  data 

Finish  all  assigned  missions.  If  systems  failure  occurs,  re-initiate  scenario. 

There  will  be  10  hours  of  “specific”  training,  followed  by  10  hours  of  performance  on 
“scenarios”,  or  assigned  missions. 

The  activities  covered  in  the  “specific”  training  will  all  be  required  in  the  scenarios 
Any  questions? 

ORIENTATION  TO  THE  SIMULATOR  (20  minutes) 

(General  Instructions) 

The  learning  goal  here  will  be  to  give  you  a  general  familiarization  with  the  simulator. 
The  basic  flight  instruments  will  be  explained,  and  you  will  get  a  chance  to  see  the 
simulator  being  flown  by  the  instructor. 

INITIAL  PRACTICE  SESSION  (30  minutes) 

This  will  be  a  general  session  in  which  you  are  to  try  to  “get  the  feel”  of  the  aircraft. 
Practice  w'ill  be  in  2  minute  segments  in  which  you  should  simply  go  to  an  altitude  of 
5000  MSL,  make  several  turns,  and  practice  control  of  the  aircraft. 
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HOUR  2  FOR  EACH  SUBJECT  rTOTAL  TIME  EST.  -  55  minutest 


TAKE-OFF  TRAINING  (24  minutes) 

TRAINING  SCENARIO  I  -  Daylight,  CAVU  conditions. 

From  dead  stop,  subject  will  initiate  AB  take-off.  Runway  heading  is  360. 

Climb  will  be  at  350  KCAS  to  1,000  MSL.  Raise  landing  gear  when  safely 
airborne.  Select  MIL  power  at  250  KCAS.  Turn  to  heading  270  at  1 ,000  MSL 
and  continue  MIL  power  climb  to  5,000  MSL.  Terminate  at  2  min.  into  exercise. 
1  min  rest.  Repeat  twice. 

TRAfNlNG  SCENARIO  2  -  Daylight,  low  vis,  low  ceiling. 

From  dead  stop,  subject  will  initiate  AB  take-off.  Runway  heading  is  360. 

Climb  will  be  at  350  KCAS  to  1,000  MSL.  Raise  landing  gear  when  safely 
airborne.  Select  MIL  power  at  250  KCAS.  Turn  to  heading  270  at  1,000  MSL 
and  continue  MIL  power  climb  to  5,000  MSL.  Terminate  at  2  min.  into  exercise. 
1  min.  rest.  Repeat  twice. 

REPEAT  SCENARIOS  1  AND  2  (TWICE  EACH) 

NAVIGATION  PRACTICE  (25  minutes) 

Instruction  on  instruments  and  mission  -  15  minutes 

The  learning  goal  here  will  be  to  assure  that  you  understand  the  navigation  instruments, 
and  that  you  can  maneuver  the  aircraft  from  one  point  to  another  with  changes  in 
altitude. 

During  this  hour,  you  will  do  one  simple  navigation  exercise,  involving  the  waypoint 
indicator.  Waypoints  will  have  been  pre-programmed,  and  you  are  simply  to  fly  to  them. 

SCENARIO  #3  -  NIGHT  NAVIGATION 

Aircraft  will  start  at  350  KCAS/5,000  MSL,  heading  270.  WPl  will  be  the  dam. 

Student  will  select  WPl,  and  will  turn  to  required  heading,  while  executing  350  KCAS 
MIL  power  climb  to  avoid  mountains.  Level  off  at  20,000  MSL,  maintaining  350 
KCAS.  At  WPl,  student  will  select  WP2  (Air  Base  #2),  and  turn  to  required  heading. 

At  1 0  nm  prior  to  WP2,  student  will  descend  to  be  level  at  500  AGL,  and  adjust  speed  to 
arrive  at  WP2  at  time  7  min.  Exercise  will  terminate  when  WP2  is  reached,  or  after  8 
min.  (Feedback  -  2  minutes) 


HOUR  3  FOR  EACH  SUBJECT  TTOTAL  TIME  EST.  =  57  minutes! 

(Continuation  ofNavigation  Training) 

TRAINING  SCENARIO  4  -  Waypoint  navigation  with  altitude  change. 

Aircraft  will  start  at  350  KCAS/5,000  MSL,  heading  270.  WPl  will  be  Factory  #2. 
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Student  will  learn  how  WP  information  would  be  entered.  Immediately  upon  scenario 
initiation,  student  will  turn  to  required  heading,  while  executing  MIL  power  climb  ai:  350 
KCAS  to  level  at  25,000  MSL.  At  WPl,  student  will  select  WP2  (bridge  #2),  and  turn  to 
required  heading.  Exercise  will  terminate  when  heading  is  achieved,  or  after  10  min. 

AIR  TO  AIR  ATTACK  TRAINING  (Total  time  =  37  minutes) 

The  goal  of  this  practice  will  be  to  provide  intensive  training  on  four  specific  air-to-air 
combat  air  patrol  engagements  in  a  night  environment.  Subjects  will  be  given  several 
practice  sessions  on  each  of  the  four  air-to-air  conditions.  They  should  develop  an 
“instinctive”  response  to  each  of  the  conditions,  and  respond  as  rapidly  as  possible  whenever 
they  recognize  that  a  specific  condition  has  occurred. 

The  following  rules  of  engagement  should  be  reviewed  with  the  subject  prior  to  entering  into 
this  phase  of  the  training  (10  minutes). 

ENGAGEMENT  RULES: 


When  any  target  is  detected  within  40  nm  West  of  WPl,  assume  to  be  hostile  aircraft. 
Bombers  will  be  below  500  AGL  and  will  maintain  their  original  formation. 

Fighters  may  be  at  any  altitude,  and  may  change  their  formation  and/or  altitude. 


Determine  which  targets  are  bombers  and  which  are  fighters.  In  the  absence  of  fighter  activity  or 
enemy  radar  lock-on,  obtain  a  Primary  Designated  Target  (PDT)  on  the  nearest  bomber  and 
launch  a  Medium-Range  Missile  (MRM)  at  maximum  range  using  heading  and  range  guidance 
provided  on  radar  and  HUD.  Attack  each  bomber  in  order  of  range,  maintaining  450 
KCAS/25,000  MSL  during  attacks.  Continue  to  monitor  all  targets  on  radar  until  last  MRM  has 
achieved  ACTIVE  status.  Then  select  MIL  and  perform  a  5G  level  turn  in  the  closest  direction 
to  WPl,  maintaining  25,000  MSL. 

If  enemy  fighters  are  detected,  monitor  their  maneuvers.  Adjust  heading  to  keep  all  enemy 
fighters  on  the  same  side  of  the  aircraft’s  nose.  If  enemy  radar  lock-on  is  detected  inside  lethal 
firing  range,  abort  attack  by  selecting  AB  and  performing  a  5G  level  turn  in  the  closest  direction 
to  WPl.  Maintain  25,000  MSL  and  dispense  2  bundles  of  chaff  when  any  hostile  fighter  with  a 
radar  lock  passes  through  the  beam  (i.e.,  90  deg  off  aircraft’s  nose/tail). 

The  following  exercises  will  be  repeated  over  the  next  25  scenarios. 

(Exercise  1  -  all  aircraft  low  -  rear  two  targets  maneuver  after  detection) 

Student  should  abort  attack  and  perform  required  maneuvers  -  4  minutes  +  1  minute  rest 

Initiate  aircraft  on  outbound  leg  of  CAP  mission,  heading  270,  350  KCAS/25,000  MSL. 
After  5  sec,  radar  will  detect  four  enemy  aircraft.  All  enemy  will  be  at  the  same  low 
altitude,  with  two  in  front  and  two  1  nm  behind.  Immediately,  rear  two  targets  separate 
into  a  “pincer  maneuver”  and  radar  lock  is  detected  on  RWR.  Student  should  abort 
attack  by  selecting  AB  and  performing  5G  level  turn  in  the  direction  closest  to  WPl. 
Maintain  25,000  MSL  and  dispense  2  bundles  of  chaff  if  any  enemy  fighter  with  a  radar 
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lock  passes  through  the  beam  (i.e.,  90  deg  off  aircraft’s  nose  or  tail).  Exercise  will 
terminate  with  release  of  chaff,  or  after  4  minutes. 

(Exercise  2  -  all  aircraft  low  -  no  maneuvering  by  any  aircraft) 

Student  should  attack  each  target  in  order  of  range  -  4  minutes  +  1  min  rest 

Initiate  aircraft  on  outbound  leg  of  CAP  mission,  heading  270,  350  KCAS/25,000  MSL. 
After  5  sec,  radar  will  detect  four  enemy  aircraft.  All  enemy  will  be  at  the  same  low 
altitude,  with  two  in  front  and  two  5  nm  behind.  No  maneuvering  or  lock-on  is  detected. 
Student  should  obtain  a  Primary  Designated  Target  (PDT)  on  the  nearest  bomber  and 
launch  a  medium-range  missile  (MRM)  at  maximum  range  using  heading  and  range 
guidance  provided  on  radar  and  HUD.  As  long  as  there  is  no  threatening  movement, 
student  should  attack  each  target  in  order  of  range.  Maintain  450  KCAS/25,000  MSL 
during  attacks.  Maintain  radar  contact  on  all  targets  until  all  MRMs  are  active,  then 
select  AB  and  perform  a  5G  turn  toward  WPl .  Exercise  will  tenninate  upon  achieving 
this  heading,  or  after  4  min. 

(Exercise  3  -  two  aircraft  low  and  ahead,  two  high  and  behind  -  no  lock  on) 

Student  should  attack  front  bombers  as  long  as  there  is  no  lock  on  -  3  minutes  +  1  min  rest 

Initiate  aircraft  on  outbound  leg  of  CAP  mission,  heading  270,  350  KCAS/25,000  MSL. 
After  5  sec,  radar  will  detect  four  enemy  aircraft.  Two  targets  will  be  low,  and  5  nm 
ahead  of  two  high  targets.  No  maneuvering  or  lock-on  is  detected.  Student  should 
obtain  a  Primary  Designated  Target  (PDT)  on  the  nearest  bomber  (low  targets)  and 
launch  a  medium-range  missile  (MRM)  at  maximum  range  using  heading  and  range 
guidance  provided  on  radar  and  HUD.  Attack  each  bomber  (low  target)  in  order  of 
range.  Maintain  450  KCAS/25,000  MSL  during  attacks.  As  long  as  there  is  no 
threatening  action  by  enemy,  continue  attacking  targets  in  order  of  distance.  Maintain 
radar  contact  with  all  targets  until  MRMs  are  active,  then  select  AB  and  perform  a  5G 
turn  toward  WP 1 . 

(Exercise  4  -  two  aircraft  high  and  in  front,  two  low  and  behind  -  fighter  lock  on) 

Student  should  perform  max  exit  from  area  -  2  minutes  +  1  min  rest 

Initiate  aircraft  on  outbound  leg  of  CAP  mission,  heading  270,  350  KCAS/25,000  MSL. 
After  5  sec,  radar  will  detect  four  enemy  aircraft.  Two  targets  will  be  high,  and  5  nm 
ahead  of  two  low  targets.  Immediately,  a  “lock-on”  warning  is  received,  with  no 
maneuvering  by  the  enemy.  Student  should  select  AB  and  perform  a  5G  turn  toward 
WP  1 .  Student  will  maintain  25,000  MSL  and  dispense  2  bundles  of  chaff  if  any  enemy 
fighter  with  a  radar  lock  passes  through  the  beam  (i.e.,  90  deg  off  aircraft’s  nose  or  tail). 
Exercise  will  terminate  with  release  of  chaff,  or  after  2  minutes. 

SCENARIO  5:  EXERCISE  #1 
SCENARIO  6:  EXERCISE  #2 
SCENARIO  7:  EXERCISE  #3 
SCENARIO  8:  EXERCISE  #1 
SCENARIO  9:  EXERCISE  #4 
SCENARIO  10:  EXERCISE#! 

SCENARIO  1 1 :  EXERCISE  #4 
SCENARIO  12;  EXERCISE  #3 
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SCENARIO  13:  EXERCISE  #4 


HOUR  FOUR  FOR  EACH  SUBJECT  (TOTAL  TIME  ESTIMATE  =  58  minutes) 


(Continuation  of  Air  to  Air  scenarios)  -  48  minutes 

SCENARIO  14:  EXERCISE  #3 
SCENARIO  15:  EXERCISE  #3 
SCENARIO  16:  EXERCISE  #2 
SCENARIO  17:  EXERCISE#! 

SCENARIO  18:  EXERCISE  #2 
SCENARIO  19:  EXERCISE#! 

SCENARIO  20:  EXERCISE  #2 
SCENARIO  2 1 :  EXERCISE  #4 
SCENARIO  22:  EXERCISE  #3 
SCENARIO  23:  EXERCISE  #4 
SCENARIO  24:  EXERCISE  #2 

“S-TURN  INSTRUCTION  -  NO  PRACTICE  (10  minutes) 

The  learning  objective  here  is  to  achieve  skill  with  this  aircraft  simulation  in  performing 
basic  flight  maneuvers  which  are  expected  of  Air  Force  pilots.  The  maneuvers  required  are 
taken  from  the  basic  Air  Force  UPT  training  syllabus. 


(Video  instruction  describing  the  S-turns) 


HOUR  5  FOR  EACH  SUBJECT  (TOTAL  TIME  ESTIMATE  =  60  minutes) 


S-TURN  PRACTICE 


Each  of  the  four  S-tum  maneuvers  will  be  done  three  times  (in  succession).  Each  one  will  be 
initiated  at  a  critical  point  where  the  pilot  will  have  to  perform  the  maneuver.  The  maneuver 
should  not  take  much  more  than  4  minutes  to  perform.  Therefore,  the  four  S-tums,  with  two 
repetitions  of  each,  and  feedback  to  the  subject,  should  not  take  more  than  48  minutes. 


SCENARIOS  #25,  26,  27) 
SCENARIOS  #28,  29,  30) 
SCENARIOS  #31,32,  33) 
SCENARIOS  #34,  35,  36) 


S  -  TURN  #  1  (repeat  three  times) 
S  -  TURN  #  2  (repeat  three  times) 
S  -  TURN  #  3  (repeat  three  times) 
S  -  TURN  #  4  (repeat  three  times) 


LANDING  INSTRUCTIONS  -  NO  PRACTICE  (12  minutes) 

The  learning  goal  here  will  be  to  give  you  the  basic  instructions  on  both  visual  and  ILS 
landing. 

(Video  instructions  -  12  minutes) 
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HOUR  SIX  FOR  ALL  SUBJECTS  (TOTAL  TIME  ESTIMATE  =  54  minutes) 

LANDING  PRACTICE 

Visual  Landing  Practice  (24  minutes) 

SCENARIO  #37  Aircraft  Visual  approach,  no  crosswinds. 

Aircraft  will  initiate  at  350  KCAS/1 0,000  MSL,  heading  090,  at  10  nm  South  and  10  nm 
West  of  runway.  WPl  will  be  10  nm  North  of  runway.  Student  will  initiate  a  300  KC.^S 
descent  to  2,000  MSL.  At  WPl,  student  will  turn  directly  to  WPO  (runway), 
maintaining  2,000  MSL  and  300  KCAS  until  achieving  a  3-deg  glidepath  to  runway 
threshold.  Deploy  speedbrakes,  lower  landing  gear,  and  perform  a  straight-in  approach 
and  landing  at  1 1  deg  angle-of  -attack  (AOA).  Student  will  not  exceed  15  deg  AOA  at 
touchdown.  Exercise  will  terminate  with  landing,  or  after  6  min.  into  the  mission. 

Allow  2  min.  for  rest. 

SCENARIO  #38  REPEAT  ABOVE  SCENARIO  (8  minutes) 

SCENARIO  #39  REPEAT  ABOVE  SCENARIO  (8  minutes) 

ILS  Landing  Practice  (30  minutes) 

(Practice  of  the  ILS  landing,  using  flight  director,  with  some  wind  gusts  -  6  minutes) 

SCENARIO  #  40  -  Aircraft  will  initiate  at  350  KCAS/10,000  MSL,  heading  090,  at  10 
nm  North  and  10  nm  West  of  runway.  WPl  will  be  10  nm  North  of  runway.  Student 
will  intercept  flight  path  and  adjust  heading  as  necessary  to  follow  flight-director 
guidance  to  intercept  the  ILS  course.  Maintain  2,000  MSL  and  300  KCAS  until 
intercepting  ILS  glideslope.  Deploy  speedbrakes,  lower  landing  gear,  and  perform  an 
ILS  flight-director  approach  at  1 1  deg  AOA. 

Upon  reaching  200  AGL,  perform  a  visual  landing.  Do  not  exceed  1 5  deg  AOA  at 
touchdown. 

Exercise  will  terminate  with  landing,  or  after  6  min.  into  the  mission.  (2  min.  rest) 
SCENARIO  #  40  -  REPEAT  ABOVE  SCENARIO  (8  minutes) 

SCENARIO  #  41  -  REPEAT  ABOVE  SCENARIO  (8  minutes) 

SCENARIO  #  42  -  REPEAT  ABOVE  SCENARIO  (8  minutes) 

SCENARIO  #  43  -  REPEAT  ABOVE  SCENARIO  (6  minutes) 
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HOUR  SEVEN  FOR  EACH  SUBJECT 


CONTINUATION  OF  ILS  APPROACH  PRACTICE  (TOTAL  TIME  ESTIMATE  =  55  minutes) 

The  learning  objective  here  is  to  achieve  skill  in  the  ILS  approach  without  the  aid  of  the 
flight  director. 


(Re-instruction  in  landing  without  flight  director,  and  on  mission  -3  minutes) 

SCENARIO  #44  -  Aircraft  will  initiate  at  350  KCAS/1 0,000  MSL,  heading  090,  at  10 
nm  North  and  10  nm  West  of  runway.  WPl  will  be  10  nm  North  of  runway.  Student 
will  intercept  the  ILS  course  with  a  maximum  intercept  angle  of  45  deg.  Maintain  2,000 
MSL  and  300  KCAS  until  intercepting  ILS  glideslope.  Deploy  speedbrakes,  lower 
landing  gear,  and  perform  an  ILS  raw-data  approach  at  1 1  deg  AOA.  Upon  reaching  200 
AGL,  perform  a  visual  landing.  Do  not  exceed  15  deg  AOA  at  touchdown.  Exercise 
will  terminate  with  landing,  or  after  1 0  min.  into  the  mission.  (Rest  1  min.) 

SCENARIO  #  45  -  REPEAT  ABOVE  SCENARIO  (1 1  minutes) 

SCENARIO  #  46  -  REPEAT  ABOVE  SCENARIO  (1 1  minutes) 

SCENARIO  #  47  -  REPEAT  ABOVE  SCENARIO  (1 1  minutes) 

SCENARIO  #  48  -  REPEAT  ABOVE  SCENARIO  (1 1  minutes) 


HOUR  EIGHT  FOR  ALL  SUBJECTS  (TOTAL  TIME  ESTIMATE  =  57  minutes) 

AIR-TO-GROUND  ATTACK  PRACTICE  (57  Minutes) 

The  learning  objectives  here,  which  will  extend  into  the  next  hour  of  training,  involve  having 
the  subject  learn,  practice,  and  become  proficient  at  a  specific  air-to-ground  attack  maneuver. 
Practice  will  involve  both  systems  variations  and  threat  environments. 

(Video  Instruction  on  A-G  pop-up  attack  -  15  minutes) 

(Practice  scenario  under  nominal  and  no-threat  conditions  -  5  minutes) 

SCENARIO  #  49  -  Aircraft  will  initiate  at  500  AGL/540  KCAS,  7  nm  from  target,  which 
will  be  dam,  and  will  be  on  a  direct  heading  of  245  toward  it.  At  IP  (5nm)  student  will 
immediately  select  AB  and  perform  a  level,  5G  right  turn  for  30  deg.  Then  student  will 
pull  up  at  5G  to  55-deg  pitch  angle  and  maintain  this,  wings-level,  until  reaching  9,800 
AGL.  Student  will  then  immediately  roll  left  (inverted)  to  place  target  locator  line  (FLL) 
vertical  relative  to  the  HUD,  and  pull  down  toward  the  target  at  3G  (keeping  the  TLL 
aligned  vertically).  Student  will  adjust  throttle  as  necessary  to  achieve  release  speed  of 
500  KCAS. 

When  target  is  visually  identified,  student  will  position  the  Bomb  Fall  Line  over  it  and 
track  the  target.  Bombs  will  be  released  when  the  Pipper  reaches  the  target.  If  the  target 
is  not  identified  visually,  student  should  bomb  the  Target  Designator  Box  (TDB). 
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Student  will  pull-out,  wings  level,  by  applying  5G  within  2  secs  following  release,  and 
will  dispense  2  flares  during  pull-out.  When  nose  reaches  horizon,  student  will  select 
Air-to-Air  (AA)  weapons  mode  and  perform  a  level  5G  left  turn  to  designated  heading, 
descending  to  500  AGL  and  dispensing  2  flares  during  turn,  maintaining  full  MIL  power 
until  5  nm  from  target.  At  5  nm  from  target,  student  will  select  NAV  mode,  and  this  will 
signal  the  end  of  the  exercise.  (4  min.).  Rest  I  minute. 

SCENARIO  #50  -  REPEAT  ABOVE  SCENARIO  (5  minutes) 

SCENARIO  #51  -  REPEAT  ABOVE  SCENARIO  (5  minutes) 

(Instruction  on  the  above  scenario  if  the  target  is  not  visually  identified  -  bomb  TDB  -  2 
minutes.) 

SCENARIO  #52  -  REPEAT  ABOVE  SCENARIO.  This  scenario  is  the  same  as  that 
above,  but  the  target  is  not  visually  identified)  (4  minutes  plus  I  min.  rest) 

SCENARIO  #  53  -  REPEAT  ABOVE  SCENARIO  -  (no.  52) 

SCENARIO  #  54  -  REPEAT  ABOVE  SCENARIO  -  (no.  52) 

(Missile  threat  avoidance  training) 

The  learning  goal  here  focuses  on  missile  avoidance  during  performance  of  the  air-to-ground 
attack. 

There  will  be  three  practice  sessions  in  which  SAM  waming(s)  will  be  given  while  the  pilot 
is  in  the  process  of  making  an  air-to-ground  attack. 

(Video  instruction  in  missile  avoidance  and  review  of  ground  attack  procedures  -  10  minutes) 


HOUR  NINE  FOR  ALL  SUBJECTS  (TOTAL  TIME  ESTIMATE  =  56  Minutes) 
AIR-TO-GROUND  ATTACK  MISSIONS  (CONT.) 

Permit  independent  review  of  A-G  mission  with  missile  warnings.  (4  minutes) 

(Practice  mission  with  missile  warnings  -  7  minutes) 

SCENARIO  #  55  -  Aircraft  will  initiate  at  the  river,  at  500  AGL/540  KCAS.  After  30 
sec.,  SAM  warning  will  be  received  on  RWR,  outside  lethal  range.  Student  should 
maneuver  to  AP,  5  nm  from  target,  which  will  be  Base  #3  and  will  be  on  a  direct  heading 
of  270  toward  it.  Monitoring  SAM  warning,  student  will  immediately  select  Mil  Power 
and  perform  a  level,  5G  right  turn  for  30  deg.  At  this  point,  another  SAM  will  be 
indicated  within  lethal  range.  Student  will  continue  attack  by  pulling  up  at  5G  to  55-deg 
pitch  angle  and  maintaining  this,  wings-level,  until  reaching  9,800  AGL.  Student  will 
then  immediately  roll  left  (inverted)  to  place  target  locator  line  (TLL)  vertical  relative  to 
the  HUD,  and  pull  down  toward  the  target  at  3G  (keeping  the  TLL  aligned  vertically). 
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Student  will  adjust  throttle  to  idle  as  nose  passes  through  the  horizon.  Dispense  two 
bundles  of  chaff  during  each  turn. 

When  target  is  visually  identified,  student  will  position  the  Bomb  Fall  Line  over  it  and 
track  the  target.  Bombs  will  be  released  when  the  Pipper  reaches  the  target.  If  the  target 
is  not  identified  visually,  student  should  bomb  the  Target  Designator  Box. 

Student  will  pull-out,  wings  level,  by  applying  5G  within  2  secs  following  release,  and 
will  dispense  2  bundles  of  chaff  during  pull-out.  When  nose  reaches  horizon,  student 
will  select  Mil-Power,  perform  a  max-G  descending  turn  in  the  nearest  direction  to  place 
the  threat  90  deg  off  aircraft’s  nose,  while  descending  to  500  AGL.  Student  will 
dispense  2  bundles  of  chaff  when  this  heading  is  reached,  and  perform  another  max-G 
turn  directly  to  a  heading  of  090,  dispensing  another  2  bundles  of  chaff  during  this  turn. 
Exercise  will  terminate  at  this  point,  or  after  6  minutes.  Rest  1  minute. 

SCENARIO  #  56  -  REPEAT  SCENARIO  #  55  (7  minutes)  -  change  location  of  the 
threat 

SCENARIO  #  57  -  REPEAT  SCENARIO  #  55  (7  minutes)  -  change  location  of  the 
threat 

SITUATION  AWARENESS  MEASURES  TRAINING 
General  Training  (15  minutes) 

The  general  nature  of  the  SA  measures  we  are  going  to  use  should  be  explained.  These 
should  be  linked  to  the  overall  purpose  of  the  development  -  to  develop  on-line  measures  of 
SA  in  a  flight  simulator  -  for  training  and  evaluation  purposes. 

We  should  cover  the  following  types  of  SA  measures: 

The  '‘blanking”  technique  -  explain  that  the  screen  will  go  blank,  and  a 
statement/question  will  appear.  The  question  will  require  a  '‘true-false”  or  “yes-no” 
answer.  Show  the  subject  where  to  respond  to  these  questions.  There  will  be  three 
categories  of  such  questions.  One  will  simply  ask  the  subject  whether  something  is  or  is 
not  present  in  the  environment  (e.g.,  there  is  a  SAM  site  to  the  right).  The  second  will 
ask  the  subject  to  estimate  or  recall  a  distance  (e.g.,  you  are  three  miles  fromThe  target). 
The  third  will  require  the  subject  to  estimate  a  future  situation  (e.g.,  you  have  to  turn 
right  in  less  than  one  minute). 

Tell  the  subject  that,  except  as  noted  below,  the  simulation  will  stop  during  these 
questions.  The  subject’s  “score”  will  be  based  on  the  speed  of  the  response  as  well  as  its 
accuracy. 

The  spatial  disorientation  technique  -  explain  that  -  occasionally  -  when  the  screen 
returns  after  one  of  the  above  questions,  the  aircraft  may  be  in  an  unusual  attitude.  The 
subject’s  task  is  to  recover  from  the  unusual  attitude  as  quickly  as  possible,  and  regain 
the  assigned  altitude,  heading,  and  airspeed. 

The  minimal  information  technique  -  this  will  require  the  most  explanation  to  the 
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subject.  Explain  that  we  are  interested  in  how  the  person  develops  a  “mental  model”  of 
the  flight  environment,  and  how  much  confidence  the  person  has  in  that  model  (i.e.,  how 
confident  that  the  model  will  predict  event  in  the  near  future).  Therefore,  we  want  to 
subject  to  fly  certain  mission  segments  with  as  little  use  of  the  displays  as  possible.  The 
displays  (and  out-the-window  views)  should  be  used  whenever  it  is  necessary  to 
“update”  the  information  the  person  believes  is  needed.  However,  this  should  be  done 
only  when  such  information  is  absolutely  required  for  safe  flight  and  mission 
accomplishment.  In  other  words,  we  are  asking  the  person  to  safely  accomplish  the 
mission  with  as  little  information  as  necessary! 

We  should  explain  that  the  final  “score”  will  depend  on  how  well  the  person  flys,  and 
also  on  how  little  they  have  to  view  the  information.  The  best  score  will  be  to  use  the 
displays  less  than  anyone  else  and  fly  the  mission  to  near  perfection.  The  worst  score 
would  be  to  crash  or  perform  the  mission  badly  (or  not  at  all)  because  the  person  used 
the  displays  too  little.  Conversely,  if  the  person  flys  very  well,  but  doesn’t  turn  the 
displays  off  at  all,  the  score  will  be  acceptable,  but  low.  Assure  the  subject  that  we  know 
this  is  not  a  realistic  situation  for  the  pilot,  but  that  the  information  we  can  get  from  this 
kind  of  probe  concerning  the  pilot’s  “mental  model”  makes  the  approach  worthwhile. 

Practice  with  the  “blanking  technique” 

(Use  training  scenario  #5  -  VISUAL  APPROACH  LANDING  -  to  demonstrate  the  three 
types  of  “blanking”  questions.) 

SCENARIO  #  58  -  The  foundation  for  this  will  be  TRAINING  SCENARIO  #  37  - 
Aircraft  Visual  approach,  very  mild  crosswinds.  Put  the  following  SA  questions  in  the 
scenario. 

Aircraft  will  initiate  at  350  KCAS/1 0,000  MSL,  heading  090,  at  10  nm  South  and  10  nm 
West  of  runway.  WPl  will  be  10  nm  North  of  runway. 

Fifteen  seconds  after  scenario  initiation: 

“The  runway  is  on  your  left”  (ANS  -  NO) 

Student  will  deploy  speedbrakes  and  initiate  a  300  KCAS  descent,  to  2,000  MSL. 

As  aircraft  passes  5,000  MSL: 

“You  are  above  5,500  MSL”  (ANS  --  NO) 

After  achieving  2,000  MSL,  but  before  waypoint  #1 : 

“Your  next  action  will  be  a  left  turn”  (ANS  --  NO) 

At  WPl,  student  will  turn  directly  to  WPO  (runway),  lower  landing  gear  and  slow  to  250 
KCAS,  maintaining  2000  MSL  until  achieving  a  3-deg  glidepath  to  runway  threshold. 

Deploy  speedbrakes,  and  perform  a  straight-in  approach  and  landing  at  1 1  deg  angle -of- 
attack  (AOA).  Student  will  not  exceed  15  deg  AOA  at  touchdown.  Exercise  will 
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terminate  with  landing,  or  after  6  min.  into  the  mission. 

(Video  demonstration  of  unusual  attitude  recovery  techniques.  10  minutes) 


HOUR  10  FOR  EACH  SUBJECT  (TOTAL  TIME  ESTIMATE  =  50  minutes) 

CONTINUATION  OF  SITUATION  AWARENESS  MEASURES  TRAINING 

The  purpose  of  this  hour  of  training  is  to  give  the  subject  experience  and  feedback  on  two  of 
the  SA  measures  -  unusual  attitude  recovery  and  minimum  information.  The  strategy  in  each 
case  is  to  give  a  brief  re-introduction  to  the  techniques,  and  then  to  give  a  maximum  amount 
of  practice  during  the  hour. 

Unusual  Attitude  Recovery  Training  (36  minutes) 

(For  all  of  these  practice  sessions,  we  will  use  TRAINING  SCENARIO  #  4  as  the  basic 
vehicle.)  Specifically,  after  the  subject  turns  to  WP  #  1  and  at  various  times  during  the  climb 
to  25,000  MSL,  we  should  momentarily  blank  the  screen  (1  to  2  seconds  -  or  whatever  time 
is  required  for  us  to  re-set  the  aircraft’s  position),  and  then  present  the  altered  attitude.  We 
estimate  that  each  presentation  should  occupy  no  more  than  6  minutes.  We  will  use  three 
basic  unusual  attitudes. 

(THE  UNUSUAL  ATTITUDES  ARE  DEFINED  IN  APPENDIX  7) 

SCENARIO  #  59  -  UNUSUAL  ATTITUDE  #  1 
SCENARIO  #  60  -  UNUSUAL  ATTITUDE  #  2 
SCENARIO  #  61  -  UNUSUAL  ATTITUDE  #  3 
SCENARIO  #  62  -  UNUSUAL  ATTITUDE  #  2 
SCENARIO  #  63  -  UNUSUAL  ATTITUDE  #  1 
SCENARIO  #  64  -  UNUSUAL  ATTITUDE  #  3 

Minimum  Information  Flying  Training  (25  minutes) 

The  goal  here  will  be  to  give  the  subject  a  chance  to  fly  several  missions  (which  have 
previously  been  practiced)  in  the  “minimum  information”  condition.  During  and  after  the 
practice,  we  should  monitor  the  subject  to  see  if  he  or  she  is  really  trying  to  fly  well  with 
minimum  infomiation.  Watch  out  for  the  subject  who  tries  to  keep  the  display  off  all  the 
time,  and  the  one  who  never  turns  it  off.  We  also  want  to  discourage  the  subject  who  turns 
the  display  off  repeatedly  for  less  than  a  second.  Emphasize  that  we  want  the  person  to 
constantly  consider  what  it  is  that  can  be  predicted  reasonably  in  the  next  several  seconds  (or 
more). 

The  actual  procedures  we  finally  choose  for  this  will  depend  on  whether  we  can  program  the 
“normally  on”  -  “normally  off’  -  and/or  “select  individual  displays”  modes  for  this 
procedure.  If  so,  we  may  have  to  revise  the  proposed  schedule  below.  Assuming  we  have 
only  the  “normally  on”  mode,  however,  we  can  use  the  following  schedule. 

SCENARIO  #  65  -  NIGHT  NAVIGATION.  For  this,  use  TRAINING  SCENARIO  #  3. 

(7  minutes). 
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SCENARIO  #  66  -  VISUAL  APPROACH  LANDING.  For  this,  use  TRAINING 
SCENARIO  #  37.  (6  minutes) 


SCENARIO  #  67. 
SCENARIO  #  49. 


AIR  TO  GROUND  POP  UP  ATTACK.  For  this,  use  TRAINING 
(4  minutes) 
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APPENDIX  7 


MISSION  SCENARIOS  TAFTER  TRAINING!  PROGRAMMED  IN  SAFTE 

SCENARIO  I:  AIR-TO-GROUND  ATTACK  MISSION 

Conditions:  daylight,  clear,  visibility  unlimited  (CAVU) 

Normal  afterburner  (AB)  takeoff  with  a  350  KCAS  climb  on  runway  heading  to  1,000 
MSL.  Raise  landing  gear  when  safely  airborne  and  select  MIL  power  at  250  KCAS. 

(SA  MEASURE  A1 :  At  900  MSL,  ask:  YOU  ARE  NOW  ABOVE  800  FEET. 
ANS  -  TRUE) 

Turn  direct  to  Waypoint  #1  (WPl)  and  continue  350  KCAS  MIL  power  climb  to  level  at 
25,000  MSL. 

(SA  MEASURE  B 1 :  At  1 5,000  MSL,  ask:  YOU  ARE  TO  LEVEL  OFF  AT 
20,000  FT.  ANS  -  FALSE). 

Turn  direct  to  WP2  and  maintain  350  KCAS/25,000  MSL. 

(SA  MEASURE  B2:  Immediately  after  WP2  has  been  entered,  ask:  YOUR 
NEXT  ALTITUDE  CHANGE  WILL  BE  TO  GO  TO  600  MSL.  ANS  -  FALSE 

At  WP2,  turn  direct  to  WP3  (Initial  Point  -  IP)  and  descend  to  be  level  at  500  MSL  1 0 
NM  prior  to  reaching  WP3. 

(SA  MEASURE  A2:  At  9  min.  into  the  mission,  ask:  YOU  ARE  LESS  THAN  8 
MINUTES  INTO  THIS  MISSION.  ANS  ~  FALSE) 

Adjust  speed  as  necessary  to  arrive  at  WP3  at  time  1 0:00  (10  mins  into  mission). 

(SA  MEASURE  B3:  At  9:45  min.  into  the  mission,  ask:  AFTER  YOU  REACH 
WP3,  YOU  SHOULD  DESCEND  TO  540  MSL.  ANS  -  FALSE 

At  the  IP  select  Air-Ground  (AG)  weapons  mode,  turn  directly  to  WP4  (Target),  and 
accelerate  to  540  KCAS  while  maintaining  500  MSL. 

(SA  MEASURE  A3:  At  5  NM  from  the  target,  ask:  YOU  SHOULD  SELECT 
ABNOW!  ANS -FALSE 

At  5  NM  from  Target  (WP4),  also  Action  Point  (AP),  select  MIL-POWER  and  perform  a 
level,  5G  right  turn  for  30  deg.  Immediately  pull  up  at  5G  to  55-deg  pitch  angle. 

Maintain  55-deg  pitch  angle  and  wings-Ievel  attitude  until  reaching  9,800  MSL. 

(SA  MEASURE  Cl :  During  ascent  (at  any  point)  ask:  ROLLING  LEFT  IS  THE 
QUICKEST  WAY  TO  BRING  THE  TARGET  VERTICAL  RELATIVE  TO 
THE  HUD.  ANS -TRUE) 
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Immediately  roll  left  (inverted)  to  place  the  Target  Locator  Line  (TLL)  vertical  relative 
to  the  HUD  and  pull  down  toward  the  target  at  3G,  keeping  the  TLL  aligned  vertically. 

(SA  MEASURE  B4:  Immediately  after  subject  begins  the  pull-down  maneuver, 
ask:  YOUR  FINAL  RELEASE  ALTITUDE  SHOULD  BE  10,000  MSL.  ANS  - 
FALSE) 

Pull  power  to  idle  as  the  nose  comes  through  the  horizon. 

When  the  Target  is  visually  identified,  position  the  Bomb  Fall  Line  over  the  Target  and 
track  the  Target.  Release  the  bombs  (Pickle  Button)  when  the  Pipper  reaches  the  Target. 
If  the  Target  is  not  identified,  bomb  the  Target  Designator  Box. 

(SA  MEASURE  B5:  If  the  altitude  goes  below  7,800  MSL,  ask:  YOU  SHOULD 
PICKLE  AS  SOON  AS  THE  PIPPER  REACHES  THE  TARGET.  ANS  - 
FALSE)  If  altitude  does  not  go  below  7,800  MSL,  skip  this  question. 

Abort  the  attack  if  a  release  is  not  performed  prior  to  reaching  8,000  MSL. 

Pull-out  wings  level  by  applying  5G  within  2  secs  following  release.  Dispense  2  flares 
during  pull-out.  When  nose  reaches  horizon,  select  Air-to-Air  weapons  mode  and 
perform  a  descending  5G  left  turn  to  a  heading  of  090  (East)  and  descend  to  500  MSL, 
dispensing  2  flares  during  turn.  Select  MIL-POWER  when  level  after  the  pullout,  and 
maintain  full  MIL-POWER  until  5  NM  from  Target. 

(SA  MEASURE  A4:  Immediately  after  aircraft  has  descended  to  500  MSL  (or 
approximate)  ask:  THERE  ARE  NO  ACTIVE  SAMS  TRACKING  YOU.  ANS  - 
-  TRUE) 

At  5  NM  from  Target  select  NAV  mode,  turn  directly  to  WP5,  and  perform  a  MIL  power 
climb  at  350  KCAS  to  25,000  MSL. 

(SA  MEASURE  B5:  At  5  NM  prior  to  WP  5,  ask:  YOUR  NEXT  TASK  WILL 
BE  TO  TURN  TO  WP  6  AND  MAINTAIN  ALTITUDE.  ANS  -  FALSE) 

At  WPS  begin  a  300  KCAS  descent  to  2,000  MSL  and  turn  directly  to  WP6. 

(SA  MEASURE  A5:  As  aircraft  passes  through  10,000  MSL,  ask:  YOU  ARE 
NOW  BELOW  10,000  MSL.  ANS  -  TRUE) 

At  WP6  turn  directly  to  WPO  (runway).  Maintain  2,000  MSL  and  300  KCAS  until 
achieving  a  3-deg  glidepath  to  runway  threshold,  then  deploy  speedbrakes,  lower  landing 
gear,  and  perform  a  straight-in  approach  and  landing  at  1 1  deg  angle-of-attack  (AOi4.). 

Do  not  exceed  1 5  deg  AOA  at  touchdown. 
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SCENARIO  2:  COMBAT  AIR  PATROL  (CAP)  MISSION 

Conditions;  night,  clear,  visibility  unlimited  (CAVU) 

Normal  afterburner  (AB)  takeoff  with  a  350  KCAS  climb  on  runway  heading  to  1,000 
MSL. 


(SA  MEASURE  B6:  During  takeoff  roll,  ask:  YOUR  CLIMB  AIRSPEED  V/ILL 
BE  350  KCAS.  ANS-TRUE) 

Raise  landing  gear  when  safely  airborne  and  select  MIL  power  at  250  KCAS. 


Turn  direct  to  Waypoint  #1  (WPl)  and  continue  350  KCAS  MIL  power  climb  to  level  at 
25,000  MSL. 

(SA  MEASURE  A6:  PRIOR  TO  WP#L  ask:  YOU  SHOULD  HAVE 
ALREADY  SELECTED  AIR-TO-AIR  MODE.  ANS  -  FALSE) 

At  WPl  select  Air-to-Air  (AA)  weapons  mode  and  turn  to  a  heading  of  270  (West)  for 
outbound  leg  of  CAP  pattern. 

(SA  MEASURE  B6;  When  aircraft  is  5  NM  from  WPl,  ask:  YOU  WILL  MAKE 
A  RIGHT  TURN  IN  3  NM.  ANS  ~  FALSE) 

When  1 0  NM  from  WP 1 ,  make  a  3G  level  left  turn  to  a  heading  of  090  (East), 
maintaining  350  KCAS/25,000  MSL  for  inbound  leg  of  CAP  pattern.  When  abeam 
WPl,  make  a  3G  level  left  turn  direct  to  WPl  and  resume  outbound  leg  of  CAP  patt(;rn, 
maintaining  350  KCAS/25,000  MSL. 


(SUBJECT  WILL  MAKE  THREE  OF  THESE  “RACETRACK”  PATTERNS  BEFORE 
ENEMY  AIRCRAFT  APPEAR) 

Monitor  radar  at  all  times  while  in  CAP. 

(SA  MEASURE  A7:  During  an  early  portion  of  the  SECOND  “outbound”  leg  of 
the  maneuver,  ask:  YOU  ARE  NOW  ON  AN  OUTBOUND  LEG  OF  THE  CAP 
PATTERN.  ANS-TRUE) 

(SA  MEASURE  Dl:  WHEN  THE  SCREEN  RE-APPEARS  AFTER  THE  LAST 
SA  QUESTION,  THE  AIRCRAFT  WILL  BE  IN  “UNUSUAL  ATTITUDE  #1”, 
AND  THE  SUBJECT  WILL  HAVE  TO  RECOVER  FROM  THIS  UNUSUAL 
ATTITUDE.)  THE  FLIGHT  WILL  THEN  CONTINUE  ACCORDING  TO 
PLAN. 

When  any  target  is  detected  West  of  WPl,  assume  to  be  hostile  aircraft.  Determine 
which  targets  are  bombers  and  which  are  fighters  (THIS  IS  SA  MEASURE  #5). 

Four  targets  appear  on  the  radar  screen  on  the  outbound  leg.  Two  aircraft  will  be  in 
front,  and  two  behind  at  approximately  5nm. 
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Obtain  a  Primary  Designated  Target  (PDT)  on  the  nearest  bomber  and  launch  a  Medium- 
Range  Missile  (MRM)  at  maximum  range  using  heading  and  range  guidance  provided  on 
radar  and  HUD. 

(SA  MEASURE  A8;  Within  5  seconds  after  the  first  launch  of  a  MRM  on  a 
bomber,  ask:  THERE  IS  NO  SAM  SITE  TARGETING  YOU  AT  THIS  TIME. 
ANS  ~  TRUE) 

Attack  each  target  identified  as  a  bomber  in  order  of  range.  Maintain  450  KCAS  du.dng 
attacks.  Continue  to  monitor  all  targets  on  radar  until  last  MRM  has  achieved  ACTIVE 
status. 


(SA  MEASURE  El:  SCHEMA  RESPONSE.) 

If  at  any  point  enemy  radar  lock-on  is  detected  inside  lethal  firing  range,  abort  attack  by 
selecting  AB  and  performing  a  5G  level  turn  in  the  closest  direction  toward  WPl . 
Maintain  25,000  MSL  and  dispense  2  bundles  of  chaff  when  any  hostile  fighter  with  a 
radar  lock  passes  through  the  beam  (i.e.,  90  deg  off  aircraft's  nose/tail). 

(SA  MEASURE  B7:  When  aircraft  reaches  2  NM  from  WPl,  ask:  THERE  ARE 
NO  ACTIVE  RADAR  OR  AIRCRAFT  THREATS  AT  THIS  TIME.  ANS  - 
TRUE) 

On  reaching  WPl  reduce  throttle  to  idle  and  deploy  speedbrakes,  select  NAV  mode,  and 
turn  directly  toward  WP2. 

(SA  MEASURE  B8:  When  speed  reaches  375  KCAS,  ask:  YOUR  DESCENT 
SPEED  WILL  BE  300  KCAS.  ANS  -  TRUE) 

Maintain  25,000  MSL  until  speed  is  reduced  to  300  KCAS,  then  descend  at  300  KC.A.S. 
Level  at  2,000  MSL/300  KCAS. 

At  WP2  turn  directly  to  WPO  (runway).  Maintain  2,000  MSL  and  250  KCAS  until 
achieving  a  3-deg  glidepath  to  runway  threshold,  then  deploy  speedbrakes,  lower  landing 
gear,  and  perform  a  straight-in  approach  and  landing  at  1 1  deg  angle-of-attack  (AOA). 

Do  not  exceed  1 5  deg  AOA  at  touchdown. 


SCENARIO  3:  (30  minutes  for  scenario  plus  rest) 

Mission:  Low-Low-Low  Air-to-Ground  Attack 
Conditions:  Daylight  clear,  visibility  unlimited  (CAVU) 

Scenario:  Normal  afterburner  (AB)  takeoff  with  a  350  KCAS  climb  on  runway  heading  to  500 
MSL. 

(SA  MEASURE  B9:  As  soon  as  aircraft  is  airborne,  ask:  YOUR  TARGET 
ALTITUDE  IS  1000  MSL.  ANS  ~  FALSE) 
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Raise  landing  gear  when  safely  airborne  and  select  MIL  power  at  250  KCAS. 

Turn  direct  to  Waypoint  #1  (WPl),  maintaining  500  MSL  and  350  KCAS. 

(SA  MEASURE  A9:  Shortly  after  speed  reaches  near  300  KCAS,  ask:  YOUR 
SPEED  IS  NOW  BELOW  350  KCAS.  ANS  -  TRUE) 

THE  NEXT  EVENT  IS  AN  UNUSUAL  ATTITUDE.  HOWEVER,  WE  HAVE 
TO  BE  SURE  THAT  THE  AIRCRAFT  HAS  ENOUGH  ALTITUDE  TO 
RECOVER  FROM  THE  UNUSUAL  ATTITUDE. 

(SA  MEASURE  D2  -  UPON  RETURN  OF  THE  SCREEN,  THE  AIRCRAFT 
SHOULD  BE  IN  UNUSUAL  ATTITUDE  #  3) 

At  WPl  turn  direct  to  WP2.  Maintain  500  MSL  and  accelerate  to  480  KCAS. 

(SA  MEASURE  BIO:  When  aircraft  reaches  2  NM  before  WP2,  ask:  AT  WP2, 
YOU  WILL  BEGIN  YOUR  ATTACK  ON  THE  TARGET.  ANS  -  FALSE) 

At  WP2,  turn  direct  to  WP3  (Initial  Point  -  IP).  Maintain  500  MSL  and  adjust  speed  as 
necessary  to  arrive  at  WP3  at  time  10:00  (10  mins  into  mission). 

(SA  MEASURE  C2:  At  9: 1 5  minutes  into  the  mission,  ask:  YOU  SHOULD 
HAVE  LESS  THAN  1  MINUTE  LEFT  BEFORE  ARRIVING  AT  WP3.  ANS  - 
TRUE) 

At  IP  select  Air-Ground  (AG)  weapons  mode,  turn  directly  to  WP4  (Target),  and 
accelerate  to  540  KCAS  while  maintaining  500  MSL. 

(SA  MEASURE  Bll :  When  aircraft  is  7  NM  from  WP4,  ask:  YOU  ARE  NOW 
7  NM  FROM  THE  TARGET.  ANS  -  TRUE) 

At  5  NM  from  Target  (WP4),  also  Action  Point  (AP),  select  MIL-POWER  and  perform  a 
level,  5G  right  turn  for  30  deg.  Immediately  pull  up  at  5G  to  55-deg  pitch  angle. 

Maintain  55-deg  pitch  angle  and  wings-level  attitude  until  reaching  9,800  MSL. 
Immediately  roll  left  (inverted)  to  place  the  Target  Locator  Line  (TLL)  vertical  relative 
to  the  HUD  and  pull  down  toward  the  target  at  3G,  keeping  the  TLL  aligned  vertically. 
Reduce  throttle  to  idle  as  nose  passes  through  the  horizon. 

(SA  MEASURE  A1 1 :  If  the  aircraft  reaches  500  KCAS,  immediately  ask:  YOU 
ARE  NOW  AT  YOUR  BOMB  RELEASE  SPEED.  ANS  ~  TRUE) 

When  the  Target  is  visually  identified,  position  the  Bomb  Fall  Line  over  the  Target  and 
track  the  Target.  Release  the  bombs  (Pickle  Button)  when  the  Pipper  reaches  the  Target. 
If  the  Target  is  not  identified,  bomb  the  Target  Designator  Box. 

(SA  MEASURE  B 1 2:  Immediately  after  pickle,  ask:  YOUR  NEXT  ACTIONS 
ARE:  I)  TO  PULL  OUT  AT  5G,  AND  2)  SELECT  AIR-TO-AIR  WEAPONS 
MODE.  ANS  -  FALSE) 
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Abort  the  attack  a  release  is  not  performed  prior  to  reaching  8,000  MSL. 

Pull-out  wings  level  by  applying  5G  within  2  secs  following  release.  Dispense  2  flares 
during  pull-out.  When  nose  reaches  horizon,  select  Air-to-Air  weapons  mode  and 
perform  a  descending  5G  left  turn  to  a  heading  of  090  (East)  and  descend  to  500  MSL, 
dispensing  2  flares  during  turn.  Select  MIL-POWER  when  leveland  maintain  until  5  MM 
from  Target. 

(SA  MEASURE  A12:  When  aircraft  is  4  NM  from  the  target,  ask:  YOU  ARE 
NOW  5  NM  FROM  THE  TARGET.  ANS  -  FALSE) 

At  5  NM  from  Target  select  NAV  mode,  turn  directly  to  WP5,  decelerate  to  480  KCAS 
and  maintain  500  MSL. 

At  WP5  slow  to  300  KCAS,  climb  to  2,000  MSL,  and  turn  directly  to  WP6. 

(SA  MEASURE  B13:  Soon  after  aircraft  reaches  2,000  MSL,  ask:  AT  WP  6, 
YOU  WILL  BEGIN  A  STRAIGHT-IN  VISUAL  APPROACH.  ANS  -  FALSE) 

(SA  MEASURE  D3:  WHEN  SCREEN  APPEARS,  THE  AIRCRAFT  WILL  BE 
IN  UNUSUAL  ATTITUDE  #2) 

At  WP6  turn  toward  WPO  (runway),  adjusting  heading  as  necessary  to  follow  flight- 
director  guidance  to  intercept  the  ILS  course. 

(SA  MEASURE  C3:  While  the  pilot  is  following  the  flight  director,  ask:  THE 
FLIGHT  DIRECTOR  APPEARS  TO  BE  GIVING  FALSE  INFORMATION. 
ANS  -  FALSE) 

Maintain  2,000  MSL  and  250  KCAS  until  intercepting  ILS  glideslope.  Deploy 
speedbrakes,  lower  landing  gear,  and  perform  an  ILS  flight-director  approach  at  1 1  deg 
AOA.  Upon  reaching  200  AGL,  perform  a  visual  landing.  Do  not  exceed  15  deg  AOA 
at  touchdown. 


SCENARIO  4:  (30  minutes,  including  scenario  and  rest) 

Mission:  Night  Air-to-Air  Combat  Air  Patrol  (CAP) 

Conditions:  night,  clear,  visibility  unlimited  (CAVU)  for  takeoff;  low  ceiling  poor  visibility  for 
landing 

Scenario:  Normal  afterburner  (AB)  takeoff  with  a  350  KCAS  climb  on  runway  heading  to  1 ,000 
MSL.  Raise  landing  gear  when  safely  airborne  and  select  MIL  power  at  250  KCAS. 

Turn  direct  to  Waypoint  #1  (WPl)  and  continue  350  KCAS  MIL  power  climb  to  level  at 
25,000  MSL. 

(SA  MEASURE  B14:  When  aircraft  passes  through  10,000  feet,  ask;  YOU  AFlE 


no 


NOW  MORE  THAN  HALFWAY  THROUGH  YOUR  CLIMB  TO  ASSIGNED 
ALTITUDE.  ANS- FALSE) 

(SA  MEASURE  D5:  WHEN  THE  SCREEN  RETURNS,  THE  AIRCRAFT 
SHOULD  BE  IN  UNUSUAL  ATTITUDE  #  4) 

At  WPl  select  Air-to-Air  (AA)  weapons  mode  and  turn  to  a  heading  of  270  (West)  for 
outbound  leg  of  CAP  pattern. 


(SA  MEASURE  A13:  When  aircraft  is  9  NM  from  WPl,  ask:  YOU  SHOULD 
BEGIN  YOUR  LEFT  TURN  NOW.  ANS  -  FALSE) 


When  10  NM  from  WPl,  make  a  3G  level  left  turn  to  a  heading  of  090  (East), 
maintaining  350  KCAS/25,000  MSL  for  inbound  leg  of  CAP  pattern. 


When  abeam  WPl,  make  a  3G  level  left  turn  direct  to  WPl  and  resume  outbound  leg  of 
CAP  pattern,  maintaining  350  KCAS/25,000  MSL. 

THE  PILOT  WILL  THEN  DO  FOUR  OF  THESE  “RACETRACK”  PATTERNS 
BEFORE  THE  ENEMY  PLANES  APPEAR. 

ON  THE  THIRD  OUTBOUND  LEG,  JUST  BEFORE  THEY  MAKE  THEIR 
TURN  (ABOUT  9.5  NM  FROM  WPl)  A  SAM  SITE  RADAR  WARNING 
SHOULD  OCCUR  OUTSIDE  OF  LETHAL  RANGE. 

(SA  MEASURE  A 14:  After  the  radar  warning  has  disappeared  (on  the  inbound 
leg  of  the  third  racetrack),  ask:  THERE  ARE  NO  SAM  SITES  ACTIVE 
WITHIN  LETHAL  RANGE.  ANS  -  TRUE) 

(SA  MEASURE  D4:  WHEN  THE  SCREEN  RETURNS,  THE  AIRCRAFT 
WILL  BE  IN  UNUSUAL  ATTITUDE  #1 .) 

Monitor  radar  at  all  times  while  in  CAP.  When  any  target  is  detected  West  of  WPl 
within  40  NM  of  WPl,  assume  to  be  hostile  aircraft.  Determine  which  targets  are 
bombers  and  which  are  fighters. 

(SA  MEASURE  E2:  SCHEMA  MEASURE) 

ON  THE  FOURTH  OUTBOUND  LEG  (WHEN  THE  AIRCRAFT  IS  5  NM 
FROM  WPl)  FOUR  TARGETS  WILL  APPEAR  ON  THE  RADAR.  THE 
FRONT  TWO  WILL  BE  HIGH,  AND  THE  REAR  TWO  WILL  BE  LOW. 
IMMEDIATELY,  THE  FRONT  TWO  WILL  START  TO  MANEUVER  AND  A 
LOCK-ON  INSIDE  LETHAL  RANGE  WILL  OCCUR,  SO  THE  REQUIRED 
ACTION  WILL  BE  TO  TURN  AND  RUN. 

If  enemy  fighters  are  detected,  monitor  their  maneuvers.  Adjust  heading  to  keep  all 
enemy  fighters  on  same  side  of  aircraft's  nose. 


If  enemy  radar  lock-on  is  detected  inside  lethal  firing  range,  abort  attack  by  selecting  AB 
and  performing  a  5G  level  turn  in  the  closest  direction  toward  WPl. 


Ill 


(SA  MEASURE  A 14:  When  the  aircraft  is  headed  toward  WPl,  ask:  THERE 
WAS  NO  NEED  TO  DISPENSE  CHAFF  DURING  THE  LAST  TURN.  ANS  - 
FALSE) 

Maintain  25,000  MSL  and  dispense  2  bundles  of  chaff  when  any  hostile  fighter  with  a 
radar  lock  passes  through  the  beam  (i.e.,  90  deg  off  aircraft's  nose/tail). 

AN  ENEMY  AIRCRAFT  WILL  CROSS  THE  BEAM 

(SA  MEASAURE  A 15:  Fifteen  seconds  after  speed  brakes  are  deployed,  ask: 
YOU  SHOULD  ALREADY  HAVE  SELECTED  NAV  MODE.  ANS  -  FALSE) 

On  reaching  WPl  reduce  throttle  to  idle  and  deploy  speedbrakes,  select  NAV  mode,  and 
turn  directly  toward  WP2. 

Maintain  25,000  MSL  until  speed  is  reduced  to  300  KCAS,  then  retract  speedbrakes  and 
descend  at  300  KCAS.  Level  at  2,000  MSL/300  KCAS. 

At  WP2  turn  toward  WPO  (runway),  lower  landing  gear  and  deploy  speedbrakes, 
adjusting  heading  as  necessary  to  follow  flight-director  guidance  to  intercept  the  ILS 
course.  Maintain  2,000  MSL  and  300  KCAS  until  intercepting  ILS  glideslope.  Perform 
an  ILS  flight-director  approach  at  1 1  deg  AOA. 

ON  THIS  APPROACH,  THE  RUNWAY  WILL  NOT  BECOME  VISABLE  TO  THE 
PILOT  BY  200  AGL. 

If  the  runway  is  not  acquired  visually  before  the  aircraft  reaches  200  AGL,  apply  full 
MIL  power,  retract  speedbrakes  and  climb  straight  ahead. 

(SA  MEASURE  A16:  When  aircraft  reaches  1000  MSL,  ask:  YOUR  LANDING 
GEAR  IS  UP.  ANS  ~  MUST  BE  COMPARED  TO  LANDING  GEAR 
STATUS.) 

When  a  positive  rate  of  climb  is  achieved,  raise  landing  gear.  Continue  climb  at  300 
KCAS  to  2,000  MSL,  then  make  a  level  right  turn  directly  to  WP2. 

(SA  MEASURE  B15:  At  2  NM  from  WP2,  ask:  AT  WP2,  YOU  WILL  MAKE  A 
LEFT  TURN.  ANS -TRUE) 

AT  THIS  POINT,  WE  NEED  TO  ASSURE  THAT  VISABILITY  IS  ABOVE  200-300 
AGL). 

Reaching  WP2,  lower  landing  gear,  slow  to  250  knots  and  turn  right  to  a  heading  of  045 
until  5  NM  from  WP2,  then  make  a  level  left  turn  to  follow  flight-director  guidance  to 
intercept  the  ILS  course  for  another  ILS  approach. 


SCENARIO  5:  (30  minutes  total  for  scenario  and  rest) 

(SCENARIOS  9,  13,  AND  1 8,  WILL  BE  IDENTICAL  TO  THIS  ONE  -  BUT  THE  SA 
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QUESTIONS  WILL  BE  DIFFERENT). 

THE  FOLLOWING  NUMBER  OF  SA  MEASURES  ARE  ADDED  INTO  THE  SCENARIO: 

A=  17  THROUGH  20 
B=  16  THROUGH  19 
C  =  4  THROUGH  6 
D=  5  THROUGH  7 
E  =  NONE 

Scenario  “A”  Description:  This  will  be  a  “Vertical  S-B”  maneuver 

Begin  at  10,000  MSL,  heading  180  (South)  at  150  KCAS,  10  nm  from  WPl  (which  is 
directly  ahead).  Gear  and  flaps  down,  speedbrakes  deployed. 

(SA  MEASURE  A 17:  At  5  nm  from  WP  1,  ask:  YOU  ARE  NOW  LESS  THAN 
3  NM  FROM  WPl .  ANS  -  FALSE) 

(SA  MEASURE  B16:  At  .5  nm  from  WPl,  ask;  YOURNEXT  MANEUVER 
WILL  BE  A  DESCENDING  LEFT  TURN.  ANS  -  TRUE) 

At  WP  1 ,  adjust  power  as  necessary  to  establish  a  1 ,000  ft/min  rate  of  descent  while 
maintaining  1 1  deg  AOA  and  a  left  bank  of  about  30  deg. 

Descend  to  9,500  MSL,  adjusting  bank  angle  during  descent  as  required  to  achieve  a 
heading  of  360  simultaneously  with  reaching  9,500  MSL. 

Add  power  as  necessary  to  establish  a  1,000  ft/min  climb  rate  while  maintaining  1 1  deg 
AOA  and  the  left  bank  required  to  achieve  a  heading  of  1 80  simultaneously  with 
regaining  10,000  MSL. 

Level  at  10,000  MSL,  1 1  deg  AOA,  heading  180. 

Scenario  “B”  Description;  This  will  be  a  “Vertical  S-C”  maneuver 

Begin  at  10,000  MSL,  heading  180  (South)  at  150  KCAS,  10  nm  from  WPl  (which  is 
directly  ahead).  Gear  and  flaps  down,  speedbrakes  deployed. 

At  WP  1,  adjust  power  as  necessary  to  establish  a  1,000  ft/min  rate  of  descent  while 
maintaining  1 1  deg  AOA  and  a  left  bank  of  about  30  deg.  Descend  to  9,500  MSL, 
adjusting  bank  angle  during  descent  as  required  to  achieve  a  heading  of  360 
simultaneously  with  reaching  9,500  MSL. 

Add  power  as  necessary  to  establish  a  1,000  ft/min  climb  rate  while  maintaining  1 1  deg 
AOA  and  the  left  bank  required  to  achieve  a  heading  of  180  simultaneously  with 
regaining  10,000  MSL. 

On  regaining  10,000  MSL  and  heading  1 80,  repeat  above  maneuver  to  the  right. 

Establish  a  1,000  ft/min  rate  of  descent  while  maintaining  1 1  deg  AOA  and  a  right  bank 
of  about  30  deg.  Descend  to  9,500  MSL,  adjusting  bank  angle  during  descent  as  required 
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to  achieve  a  heading  of  360  simultaneously  with  reaching  9,500  MSL. 

Add  power  as  necessary  to  establish  a  1,000  ft/min  climb  rate  while  maintaining  1 1  deg 
AOA  and  the  right  bank  required  to  achieve  a  heading  of  1 80  simultaneously  with 
regaining  10,000  MSL. 

Level  at  10,000  MSL,  1 1  deg  AOA,  heading  180. 


SCENARIO  6:  (30  minutes  total  for  scenario  and  rest) 

Mission:  Night  Low-Altitude  Air-to-Air  Combat  Air  Patrol  (CAP) 

Conditions:  night,  clear,  visibility  unlimited  (CAVU) 

Scenario:  Normal  afterburner  (AB)  takeoff  with  a  350  KCAS  climb  on  runway  heading  to  1 ,000 
MSL.  Raise  landing  gear  when  safely  airborne  and  select  MIL  power  at  250  KCAS. 

(SA  MEASURE  A21:  As  soon  as  the  landing  gear  is  retracted  (if  below  350 
KCAS,  ask:  YOU  ARE  AT  OR  ABOVE  350  KCAS.  ANS  -  FALSE) 

Turn  direct  to  Waypoint  #1  (WPl)  and  continue  350  KCAS  MIL  power  climb  to  level  at 
25,000  MSL. 

(SA  MEASURE  B20:  When  aircraft  passes  through  10,000  feet,  ask:  YOU  AFLE 
NOW  MORE  THAN  HALFWAY  THROUGH  YOUR  CLIMB  TO  ASSIGNED 
ALTITUDE.  ANS -FALSE) 

(SA  MEASURE  B21 :  When  aircraft  is  1 1  NM  from  WPl,  ask:  YOU  HAVE  TO 
PERFORM  AN  ACTION  IN  1  NM.  ANS  -  TRUE) 

(SA  MEASURE  D8:  WHEN  THE  SCREEN  RETURNS,  THE  AIRCRAFT 
WILL  BE  IN  UNUSUAL  ATTITUDE  #2.) 

10  NM  prior  to  WPl  extend  speedbrakes  and  descend  at  idle  throttle  to  level  at  1,000 
MSL  prior  to  reaching  WPl .  Retract  speedbrakes  and  level  at  1,000  MSL/350  KCAS. 

(SA  MEASURE  B22:  When  speedbrakes  are  extended  OR  idle  throttle  is 
reached, 

ask:  YOUR  ASSIGNED  ALTITUDE  WILL  NOT  BE  REACHED  FOR  THREE 
MINUTES.  ANS -FALSE) 

(ON  THE  ABOVE,  THE  ACTUAL  DESCENT  RATE  IS  CHECKED  IN 
ORDER  TO  DETERMINE  WHETHER  THE  ANSWER  IS  CORRECT  OR 
NOT) 

At  WPl  select  Air-to-Air  (AA)  weapons  mode  and  turn  to  a  heading  of  270  (West)  for 
outbound  leg  of  CAP  pattern. 

(SA  MEASURE  A22:  When  aircraft  is  5  miles  from  WPl,  ask:  YOU  ARE  NOW 
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SIX  MILES  FROM  WPl.  ANS  -  FALSE) 

When  1 0  NM  from  WP 1 ,  make  a  3G  level  left  turn  to  a  heading  of  090  (East), 
maintaining  350  KCAS/1,000  MSL  for  inbound  leg  of  CAP  pattern.  When  abeam  V/Pl. 
make  a  3G  level  left  turn  direct  to  WPl  and  resume  outbound  leg  of  CAP  pattern, 
maintaining  350  KCAS/1,000  MSL. 

(SA  MEASURE  E3:  SCHEMA  MEASURE) 

ON  THIS  SCENARIO,  WE  WILL  ONLY  HAVE  THE  STUDENT  DO  ONE 
COMPLETE  “RACETRACK”.  ON  THE  SECOND  OUTBOUND  LEG,  1  NM 
FROM  WPl,  THE  FOUR  TARGETS  WILL  APPEAR  ON  THE  SCREEN.  ALL 
WILL  BE  LOW  (ALL  BOMBERS). 

Obtain  a  Primary  Designated  Target  (PDT)  on  the  nearest  bomber  and  launch  a  Medium- 
Range  Missile  (MRM)  at  maximum  range  using  heading  and  range  guidance  provided  on 
radar  and  HUD. 

(SA  MEASURE  A23;  Immediately  after  the  second  MRM  has  been  launched, 
ask:  YOUR  SECOND  MRM  IS  NOW  “ACTIVE”.  ANS  ~  TRUE 

Attack  each  bomber  in  order  of  range.  Maintain  450  KCAS/1,000  MSL  during  attacks. 

(SA  MEASURE  B23:  Immediately  after  the  third  MRM  has  been  launched,  ask: 
THE  LAST  TARGET  IS  PROBABLY  A  FIGHTER.  ANS  -  FALSE) 

Continue  to  monitor  all  targets  on  radar  until  last  MRM  has  achieved  ACTIVE  status. 
Then  select  MIL  and  perform  a  5G  level  turn  in  the  closest  direction  toward  WPl, 
maintaining  1,000  MSL. 

(SA  MEASURE  A24;  When  the  aircraft  reaches  3G,  ask:  YOU  ARE  NOW 
PULLING  MORE  THAN  4G.  ANS  ~  FALSE) 

On  reaching  WPl  perform  a  MIL-power  350  KCAS  climb  to  25,000  MSL,  select  NAV 
mode,  and  turn  directly  toward  WP2.  Maintain  25,000  MSL  until  10  NM  from  WP2, 
then  extend  speedbrakes  and  descend  at  300  KCAS  to  2,000  MSL  using  idle  throttle. 
Retract  speedbrakes  and  level  at  2,000  MSL/300  KCAS. 

(SA  MEASURE  B24:  When  aircraft  reaches  10.5  NM  from  WP2,  ask:  YOL  R 
NEXT  TARGETED  ALTITUDE  IS  2,000  MSL.  ANS  -  TRUE) 

At  WP2  turn  toward  WPO  (runway),  lower  landing  gear  and  deploy  speedbrakes, 
adjusting  heading  as  necessary  to  intercept  the  ILS  course  with  a  maximum  intercept 
angle  of  45  deg.  Maintain  2,000  MSL  and  300  KCAS  until  intercepting  ILS  glideslope. 
Perform  an  ILS  raw-data  approach  at  1 1  deg  AOA.  Upon  reaching  200  AGL,  perform  a 
visual  landing.  Do  not  exceed  15  deg  AOA  at  touchdown. 


SCENARIO  7:  (30  minutes  total  for  scenario  and  rest) 
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Conditions:  Daylight  attack  and  landing,  low  ceiling,  poor  visibility.  Ceiling  at  target  is  2.500. 


Scenario:  Normal  afterburner  (AB)  takeoff  with  a  350  KCAS  climb  on  runway  heading  to  1,000 
MSL.  Raise  landing  gear  when  safely  airborne  and  select  MIL  power  at  250  KCAS. 

Turn  direct  to  Waypoint  #1  (WPl)  and  continue  350  KCAS  MIL  power  climb  to  level  at  25,000 
MSL. 


(SA  MEASURE  A25:  When  aircraft  reaches  2 1 ,000  MSL,  ask:  YOU  ARE 
NOW  ABOVE  20,500  MSL.  ANS  -  TRUE) 

(SA  MEASURE  D9:  WHEN  SCREEN  RETURNS,  AIRCRAFT  WILL  BE  IN 
UNUSUAL  ATTITUDE  #3) 

Turn  direct  to  WP2  and  maintain  350  KCAS/25,000  MSL. 

(SA  MEASURE  B25:  When  aircraft  is  three  minutes  from  WP2,  ask:  IN  JUST 
UNDER  TWO  MINUTES,  YOU  WILL  BEGIN  TO  DESCEND.  ANS  - 
FALSE) 

At  WP2,  turn  direct  to  WP3  (Initial  Point  -  IP)  and  descend  to  be  level  at  500  MSL  10 
NM  prior  to  reaching  WP3. 

(SA  MEASURE  C7:  When  aircraft  is  3  min.  from  WP3,  ask:  YOU  HAVE 
TRAVELED  MORE  THAN  HALF  THE  DISTANCE  BETWEEN  WP2  AND 
WP3.  ANS -TRUE) 

Adjust  speed  as  necessary  to  arrive  at  WP3  at  time  10:00  (10  mins  into  mission). 

(SA  MEASURE  B26:  YOUR  NEXT  ACTION  IS  TO  CLIMB.  ANS  -  FALSE) 

At  IP  select  Air-Ground  (AG)  weapons  mode,  turn  directly  to  WP4  (Target),  and 
accelerate  to  540  KCAS  while  climbing  to  1,500  MSL. 

Continue  to  fly  directly  toward  Target  until  visual  identification  is  made.  When  the 
Target  is  visually  identified,  position  the  Bomb  Fall  Line  over  the  Target  and  track  the 
Target.  Push  over  (bunt)  into  a  shallow  (5-deg)  dive  at  release.  Release  the  bombs 
(Pickle  Button)  when  the  Pipper  reaches  the  Target. 

If  the  Target  is  not  identified,  bomb  the  Target  Designator  Box. 

(SA  MEASURE  B27:  When  the  aircraft  passes  1,300  MSL,  ask:  YOU  MUST 
ABORT  THE  ATTACK  IF  YOU  HAVE  NOT  RELEASED  THE  BOMB  BY 
100  FEET  LOWER  THAN  PRESENT  ALTITUDE.  ANS  -  FALSE) 

Abort  the  attack  if  dive  angle  (Flight  Path  Marker,  FPM)  exceeds  10  deg,  or  if  a  release 
is  not  performed  prior  to  reaching  900  MSL. 

Pull-out  wings  level  by  applying  5G  within  2  secs  following  release. 


116 


(SA  MEASURE  B28:  Immediately  after  release,  ask:  YOUR  NEXT  TURN  IS 
TO  THE  EAST.  ANS  -  TRUE) 

When  nose  reaches  horizon,  select  full  MIL  power  and  Air-to-Air  weapons  mode,  and 
perform  a  descending  5G  left  turn  to  a  heading  of  090  (East).  Dispense  2  flares  during 
turn.  Descend  to  500  MSL  and  maintain  full  MIL  power  until  5  NM  from  Target. 

(SA  MEASURE  A26:  When  aircraft  is  4.5  NM  from  target,  ask:  YOU  ARE 
NOW  5  NM  FROM  THE  TARGET.  ANS  -  FALSE) 

At  5  NM  from  Target  select  NAV  mode,  turn  directly  to  WP5,  and  perform  a  MIL  power 
climb  at  350  KCAS  to  25,000  MSL. 

(SA  MEASURE  A27:  When  aircraft  passes  22,000  MSL,  ask;  YOU  ARE  NOW 
AT  22,000  MSL.  ANS  -  TRUE) 

At  WP5  begin  a  300  KCAS  descent  to  2,000  MSL  and  turn  directly  to  WP6. 

(SA  MEASURE  C8;  When  aircraft  passes  1 1,000  MSL,  ask:  YOU  ARE  MORE 
THAN  HALFWAY  TO  YOUR  NEW  ALTITUDE.  ANS  -  TRUE) 

At  WP6  turn  toward  WPO  (runway),  lower  landing  gear  and  deploy  speedbrakes, 
adjusting  heading  as  necessary  to  intercept  the  ILS  course  with  a  maximum  intercept 
angle  of  45  deg.  Maintain  2,000  MSL  and  300  KCAS  until  intercepting  ILS  glideslope. 
Perform  an  ILS  raw-data  approach  at  1 1  deg  AOA. 

IN  THIS  SCENARIO,  THE  RUNWAY  WILL  NOT  BE  VISUALLY 
IDENTIFIED  BY  THE  TIME  THE  AIRCRAFT  REACHES  200  AGL. 

If  the  runway  is  not  acquired  visually  before  the  aircraft  reaches  200  AGL,  apply  full 
MIL  power,  retract  speedbrakes  and  climb  straight  ahead.  When  a  positive  rate  of  climb 
is  achieved,  raise  landing  gear. 

(SA  MEASURE  A28;  When  aircraft  passes  1,500  MSL,  ask:  YOU  ARE  NOW 
AT  1,200  MSL.  ANS  -  FALSE) 

(SA  MEASURE  B29:  Immediately  after  the  last  question  is  answered,  ask: 
YOUR  NEXT  GOAL  IS  TO  FLY  TO  WP5.  ANS  ~  FALSE) 

Continue  climb  at  300  KCAS  to  2,000  MSL,  then  make  a  level  right  turn  directly  to 
WP6. 


(AT  THIS  POINT,  WE  NEED  TO  ASSURE  THAT  VISABILITY  IS  ABOVE 
200-300  AGL). 

Reaching  WP6,  lower  landing  gear,  slow  to  250  KCAS,  turn  right  to  a  heading  of  045 
until  5  NM  from  WP6,  then  make  a  level  left  turn  to  a  heading  of  225  deg  to  intercept  the 
ILS  course  for  another  ILS  approach. 
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SCENARIO  8:  (30  minutes  total  for  scenario  and  rest) 


THIS  WILL  BE  A  MISSION  IN  WHICH  WE  USE  THE  “MINIMUM  INFORMATION” 
TECHNIQUE.  THE  SUBJECT  WILL  BE  INSTRUCTED  THAT  PORTIONS  OF  THE 
MISSION  ARE  TO  BE  FLOWN  WITH  ONLY  THE  DISPLAYS  WHICH  ARE  ABSOLUTELY 
ESSENTIAL  FOR  SAFETY  AND  SUCCESSFUL  MISSION  COMPLETION.  THESE 
SEGMENTS  WILL  BE  INTRODUCED  BY  HAVING  THE  SCREEN  GO  “BLANK”,  AND 
THEN  THE  FOLLOWING  MESSAGE  WILL  APPEAR  FOR  FIVE  SECONDS;  “UNTIL 
FURTHER  NOTICE,  YOU  WILL  HAVE  TO  CALL  UP  ANY  INFORMATION  YOU  NEED 
BY  CLICKING  THE  SCREEN.”  AT  THE  END  OF  THE  MINUMUM  INFORMATION 
SEGMENT,  THE  SCREEN  GOES  BLANK  MOMENTARILY,  AND  A  MESSAGE  SAYS 
(“RETURN  TO  NORMAL  FLYING  PROCEDURES”). 

Mission:  Night  Air-to-Air  Combat  Air  Patrol  (CAP) 


Conditions:  night,  clear,  visibility  unlimited  (CAVU) 

Scenario;  Normal  afterburner  (AB)  takeoff  with  a  350  KCAS  climb  on  runway  heading  to  500 
MSL.  Raise  landing  gear  when  safely  airborne  and  select  MIL  power  at  250  KCAS. 

(SA  MEASURE  FI:  START  MINIMUM  INFORMATION  SEGMENT) 

Turn  direct  to  Waypoint  #1  (WPl);  maintain  350  KCAS/500  MSL. 

30  NM  prior  to  WPl  begin  a  MIL  power  350  KCAS  climb  to  25,000  MSL. 
Level  at  25,000  MSL/350  KCAS. 


At  WPl  select  Air-to-Air  (AA)  weapons  mode. 

(END  MINIMUM  INFORMATION  SEGMENT  AS  SOON  AS  AA  MODE  IS 
SELECTED) 

Turn  to  a  heading  of  270  (West)  for  outbound  leg  of  CAP  pattern. 


(SA  MEASURE  C9:  When  aircraft  is  7  NM  from  WPl,  ask:  YOU  WILL  MAKE 
A  RIGHT  TURN  IN  ABOUT  7  NM.  ANS  -  FALSE) 

When  1 0  NM  from  WPl ,  make  a  3G  level  left  turn  to  a  heading  of  090  (East), 
maintaining  350  KCAS/25,000  MSL  for  inbound  leg  of  CAP  pattern. 

When  abeam  WPl,  make  a  3G  level  left  turn  direct  to  WPl  and  resume  outbound  leg  of 
CAP  pattern,  maintaining  350  KCAS/25,000  MSL. 

(SA  MEASURE  F2:  START  MINIMUM  INFORMATION  SEGMENT  AS 
SOON  AS  AIRCRAFT  REACHES  WPl.) 

WE  WILL  KEEP  UP  THE  MINIMUM  INFORMATION  SEGMENT  FOR 
TWO  “REVOLUTIONS”  OF  THE  RACETRACK.  IT  WILL  END  WHEIN  THE 
AIRCRAFT  REACHES  WPl  AFTER  THE  SECOND  REVOLUTION. 
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Monitor  radar  as  required  while  in  CAP.  When  any  target  is  detected  West  of  WPl 
within  40  NM  of  WPl,  assume  to  be  hostile  aircraft.  Determine  which  targets  are 
bombers  and  which  are  fighters. 

ON  THIS  SCENARIO,  THE  FOUR  TARGETS  WILL  APPEAR  WHEN  THE 
AIRCRAFT  IS  HALFWAY  OUT  ON  THE  OUTBOUND  LEG.  THE  FRONT 
TWO  TARGETS  WILL  BE  BOMBERS  (LOW)  AND  THE  REAR  TWO  WILL 
BE  FIGHTERS  (BUT  THEY  WILL  ALSO  BE  LOW).  THE  FIGHTERS  WILL 
NOT  MANEUVER  UNTIL  AFTER  THE  FRONT  TWO  BOMBERS  HAVE 
BOTH  BEEN  ATTACKED.  WE  WILL  ALSO  INTRODUCE  SAM 
WARNINGS,  BUT  THEY  WILL  BE  OUTSIDE  LETHAL  RANGE. 

(SA  MEASURE  E4:  SCHEMA  MEASURE) 

Obtain  a  Primary  Designated  Target  (PDT)  on  the  nearest  bomber  and  launch  a  Medium- 
Range  Missile  (MRM)  at  maximum  range  using  heading  and  range  guidance  provided  on 
radar  and  HUD.  Attack  each  bomber  in  order  of  range.  Maintain  450  KCAS/25,000 
MSL  during  attacks. 

AFTER  THE  FIRST  MRM  IS  LAUNCHED,  A  SAM  WARNING  WILL 
APPEAR  OUTSIDE  OF  LETHAL  RANGE. 

Continue  to  monitor  all  targets  on  radar  until  last  MRM  has  achieved  ACTIVE  status. 

AFTER  THE  SECOND  MRM  IS  LAUNCHED,  THE  TWO  FIGHTERS  WILL 
BEGIN  TO  MANEUVER,  AND  THEY  WILL  HAVE  LOCK-ON  INSIDE 
LETHAL  RANGE. 

If  enemy  fighters  are  detected,  monitor  their  maneuvers.  Adjust  heading  to  keep  all 
enemy  fighters  on  same  side  of  aircraft's  nose.  If  enemy  radar  iock-on  is  detected  inside 
lethal  firing  range,  abort  attack  by  selecting  AB  and  performing  a  5G  level  turn  in  the 
closest  direction  toward  WPl.  Maintain  25,000  MSL  and  dispense  2  bundles  of  chaff 
when  any  hostile  fighter  with  a  radar  lock  passes  through  the  beam  (i.e.,  90  deg  off 
aircraft's  nose/tail). 

(SA  MEASURE  A29:  As  soon  as  (if)  AB  is  selected,  ask:  THERE  ARE  NO 
ACTIVE  SAM  SITES  FN  THE  AREA.  ANS- FALSE) 

(SA  MEASURE  DIO:  WHEN  SCREEN  RETURNS,  THE  AIRCRAFT  WILL 
BE  IN  UNUSUAL  ATTITUDE  #4) 


At  WPl  turn  direct  to  WP2. 

(SA  MEASURE  F3:  START  MINIMUM  INFORMATION  SEGMENT  AS 
SOON  AS  WP2  IS  ENTERED) 


Extend  speedbrakes  and  descend  at  idle  power,  350  KCAS  to  500  MSL.  Retract 
speedbrakes  and  level  at  500  MSL/350  KCAS. 

5  NM  prior  to  WP2  perform  a  MIL-power  350  KCAS  climb  to  level  at  2,000  MSL. 
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At  WP2  turn  toward  WPO  (runway). 

(END  MINIMUM  INFORMATION  SEGMENT  AS  SOON  WP2  IS 
REACHED.) 

Lower  landing  gear  and  deploy  speedbrakes.  Adjust  heading  as  necessary  to  intercept 
the  ILS  course  with  a  maximum  intercept  angle  of  45  deg.  Maintain  2,000  MSL  and  300 
KCAS  until  intercepting  ILS  glideslope.  Perform  an  ILS  raw-data  approach  at  1 1  deg 
AOA.  Upon  reaching  200  AGL,  perform  a  visual  landing.  Do  not  exceed  15  deg  AOA 
at  touchdown. 


SCENARIO  9:  (30  minutes  total  for  scenario  and  rest) 

THIS  WILL  BE  THE  SECOND  S-TURN  SCENARIO.  WE  WILL  USE  THE 
FOLLOWING  SA  MEASURES  IN  THIS  SCENARIO: 

A  =  30  THROUGH  34 
B  =  30  THROUGH  33 
C=  10  THROUGH  12 
D  =  11  THROUGH  13 
E=NONE 
F  =  NONE 


Scenario  “A”  Description:  This  will  be  a  “Vertical  S-B”  maneuver 

Begin  at  10,000  MSL,  heading  180  (South)  at  150  KCAS,  10  nm  from  WPl  (which  is 
directly  ahead).  Gear  and  flaps  down,  speedbrakes  deployed. 

At  WP  I ,  adjust  power  as  necessary  to  establish  a  1 ,000  ft/min  rate  of  descent  while 
maintaining  1 1  deg  AOA  and  a  left  bank  of  about  30  deg.  Descend  to  9,500  MSL, 
adjusting  bank  angle  during  descent  as  required  to  achieve  a  heading  of  360 
simultaneously  with  reaching  9,500  MSL. 

Add  power  as  necessary  to  establish  a  1,000  ft/min  climb  rate  while  maintaining  1 1  deg 
AOA  and  the  left  bank  required  to  achieve  a  heading  of  1 80  simultaneously  with 
regaining  10,000  MSL. 

Level  at  10,000  MSL,  1 1  deg  AOA,  heading  180. 


Scenario  “B”  Description:  This  will  be  a  “Vertical  S-C”  maneuver 

Begin  at  10,000  MSL,  heading  180  (South)  at  150  KCAS,  10  nm  from  WPl  (which  is 
directly  ahead).  Gear  and  flaps  down,  speedbrakes  deployed. 

At  WP  1,  adjust  power  as  necessary  to  establish  a  1,000  ft/min  rate  of  descent  while 
maintaining  1 1  deg  AOA  and  a  left  bank  of  about  30  deg.  Descend  to  9,500  MSL, 
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adjusting  bank  angle  during  descent  as  required  to  achieve  a  heading  of  360 
simultaneously  with  reaching  9,500  MSL. 

Add  power  as  necessary  to  establish  a  1,000  ft/min  climb  rate  while  maintaining  1 1  deg 
AOA  and  the  left  bank  required  to  achieve  a  heading  of  1 80  simultaneously  with 
regaining  10,000  MSL. 

On  regaining  10,000  MSL  and  heading  180,  repeat  above  maneuver  to  the  right. 

Establish  a  1,000  ft/min  rate  of  descent  while  maintaining  1 1  deg  AOA  and  a  right  bank 
of  about  30  deg.  Descend  to  9,500  MSL,  adjusting  bank  angle  during  descent  as  required 
to  achieve  a  heading  of  360  simultaneously  with  reaching  9,500  MSL. 

Add  power  as  necessary  to  establish  a  1,000  ft/min  climb  rate  while  maintaining  1 1  deg 
AOA  and  the  right  bank  required  to  achieve  a  heading  of  1 80  simultaneously  with 
regaining  10,000  MSL. 

Level  at  10,000  MSL,  1 1  deg  AOA,  heading  180. 


SCENARIO  10 

THIS  WILL  BE  THE  SECOND  MISSION  IN  WHICH  WE  USE  THE  “MINIMUM 
INFORMATION”  TECHNIQUE.  THIS  WILL  BE  AN  AIR-TO-GROUND  MISSION.  THE 
SUBJECT  WILL  BE  INSTRUCTED  THAT  PORTIONS  OF  THE  MISSION  ARE  TO  BE 
FLOWN  WITH  ONLY  THE  DISPLAYS  WHICH  ARE  ABSOLUTELY  ESSENTIAL  FOR 
SAFETY  AND  SUCCESSFUL  MISSION  COMPLETION.  THESE  SEGMENTS  WILL  BE 
INTRODUCED  BY  HAVING  THE  SCREEN  GO  “BLANK”,  AND  THEN  THE  FOLLOWifNG 
MESSAGE  WILL  APPEAR  FOR  FIVE  SECONDS;  “  UNTIL  FURTHER  NOTICE,  YOU 
WILL  HAVE  TO  CALL  UP  ANY  INFORMATION  YOU  NEED  BY  CLICKING  THE 
SCREEN.”  AT  THE  END  OF  THE  MINIMUM  INFORMATION  SEGMENT,  THE  SCREEN 
GOES  BLANK  MOMENTARILY,  AND  A  MESSAGE  SAYS  (“RETURN  TO  NORMAL 
FLYING  PROCEDURES”). 


Mission:  Low-High-Low  Air-to-Ground  Attack 

Conditions:  Daylight  attack  and  landing,  low  ceiling,  poor  visibility  for  landing,  CAVU  for 
takeoff  and  in  target  area 

Scenario:  Normal  afterburner  (AB)  takeoff  with  a  350  KCAS  climb  on  runway  heading  to  500 
MSL.  Raise  landing  gear  when  safely  airborne  and  select  MIL  power  at  250  KCAS. 

(SA  MEASURE  F4:  START  MINIMUM  INFORMATION  SEGMENT  AS 
SOON  AS  THE  LANDING  GEAR  ARE  RETRACTED.) 

Turn  direct  to  Waypoint  #1  (WPl),  maintaining  500  MSL  and  350  KCAS. 

At  WPl  turn  direct  to  WP2  and  perform  a  MIL-power  climb  at  350  KCAS  to  25,000 
MSL.  Accelerate  to  420  KCAS  after  reaching  25,000  MSL. 

At  WP2,  turn  direct  to  WP3  (Initial  Point  -  IP)  and  descend  to  be  level  at  500  MSL  10 
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NM  prior  to  reaching  WP3.  Adjust  speed  as  necessary  to  arrive  at  WP3  at  time  1 0:00 
(10  mins  into  mission). 

At  IP  select  Air-Ground  (AG)  weapons  mode,  turn  directly  to  WP4  (Target),  and 
accelerate  to  540  KCAS  while  maintaining  500  MSL. 

(END  MINIMUM  INFORMATION  SEGMENT  AS  SOON  AS  AG  MODE  IS 
SELECTED.) 

At  5  NM  from  Target  (WP4),  also  Action  Point  (AP),  select  MIL-POWER  and  perform  a 
level,  5G  right  turn  for  30  deg. 

(SA  MEASURE  Cl 3:  When  aircraft  has  turned  15  deg.,  ask:  YOU  ARE  AT 
LEAST  HALFWAY  THROUGH  YOUR  REQUIRED  TURN.  ANS  -  TRUE) 

Immediately  pull  up  at  5G  to  55-deg  pitch  angle. 

Maintain  55-deg  pitch  angle  and  wings-Ievel  attitude  until  reaching  9,800  MSL. 

(SA  MEASURE  A35:  When  aircraft  passes  through  5,000  MSL,  ask:  YOU  AF’E 
NOW  AT  6,000  MSL.  ANS  -  FALSE) 

Immediately  roll  left  (inverted)  to  place  the  Target  Locator  Line  (TLL)  vertical  relative 
to  the  HUD  and  pull  down  toward  the  target  at  3G,  keeping  the  TLL  aligned  vertically. 
Select  idle  power  as  nose  passes  through  horizon. 

AS  SOON  AS  THE  POWER  IS  REDUCED  TO  IDLE  IN  THIS  CONDITION,  A  SAM 
WARNING  WITHIN  LETHAL  RANGE  WILL  BE  GIVEN. 

The  student  should  proceed  with  the  mission.  When  the  Target  is  visually  identified, 
position  the  Bomb  Fall  Line  over  the  Target  and  track  the  Target.  Release  the  bombs 
(Pickle  Button)  when  the  Pipper  reaches  the  Target.  If  the  Target  is  not  identified,  bomb 
the  Target  Designator  Box. 

Abort  the  attack  if  release  is  not  performed  prior  to  reaching  8,000  MSL. 

Pull-out  wings  level  by  applying  5G  within  2  secs  following  release.  Dispense  2  flares 
during  pull-out. 

After  dive  recovery,  perform  a  max-G  turn  in  nearest  direction  to  place  threat  90  deg  off 
aircraft’s  nose  while  descending  to  500  AGL. 

(SA  MEASURE  B34:  As  soon  as  a  2-G  turn  is  detected,  ask:  YOU  HAVE 
ALREADY  DISPENSED  FOUR  BUNDLES  OF  CHAFF  IN  THIS 
MANEUVER.  ANS  -  FALSE) 

Dispense  2  bundles  of  chaff  during  this  turn,  then  perform  a  max-G  turn  directly  to  a, 
heading  of  090  (East),  dispense  2  additional  chaff  bundles. 

When  nose  reaches  horizon,  select  Air-to-Air  weapons  mode  and  perform  a  level  5G  left 
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turn  to  a  heading  of  090  (East)  and  descend  to  500  MSL,  dispensing  2  flares  during  turn. 

(SA  MEASURE  B35:  When  the  aircraft  is  3  NM  from  the  target,  ask:  YOU 
SHOULD  HAVE  CHANGED  THE  NAV  MODE  BY  THIS  TIME.  ANS  - 
FALSE) 

Maintain  full  MIL  power  until  5  NM  from  Target. 

At  5  NM  from  Target  select  NAV  mode,  turn  directly  to  WPS,  decelerate  to  480  KCAS 
and  maintain  500  MSL. 

At  WPS  slow  to  300  KCAS,  climb  to  2,000  MSL,  and  turn  directly  to  WP6. 

At  WP6  turn  toward  WPO  (runway),  lower  landing  gear  and  deploy  speedbrakes, 
adjusting  heading  as  necessary  to  follow  flight-director  guidance  to  intercept  the  ILS 
course.  Maintain  2,000  MSL  and  300  KCAS  until  intercepting  ILS  glideslope.  Perform 
an  ILS  flight-director  approach  at  1 1  deg  AO  A. 

(AT  THIS  POINT,  WE  NEED  TO  ASSURE  THAT  VISABILITY  IS  ABOVE 
200-300  AGL). 

When  the  runway  is  visually  identified,  perform  a  visual  landing.  Do  not  exceed  1 5  deg 
AOA  at  touchdown. 

FOR  SCENARIOS  1 1  TO  20,  REPEAT  THE  ABOVE  10  IN  A  RANDOM  ORDER. 
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APPENDIX  8 


THE  DATA  GENERATOR  FOR  SAFTE 


The  raw  data  are  broken  down  into  twelve  distinct  types  of  events.  Whenever  any  one  of  these 
events  occurs,  it  is  written  as  a  single  record.  Detection  of  these  events  is  based  on  measuring 
changes  in  the  readings  (which  are  sampled  at  varying  rates  for  different  types  of  data  -  ranging 
from  5/sec  for  user  input  values  to  1/sec  for  aircraft  state  values.)  Detection  occurs  when  a 
threshold  trip  value  is  surpassed  and  ends  when  the  event  fails  to  exceed  a  threshold  stop  value. 
Written  into  the  program  is  the  ability  to  change  the  threshold  levels  for  both  the  trip  value  and 
the  stop  value.  The  trip  value  sets  the  minimum  value  a  variable  must  change  to  initiate  a  record. 
The  stop  value  sets  the  maximum  value  a  variable  can  change  to  end  a  record.  For  example,  if 
the  threshold  values  for  an  altitude  change  are:  Trip  =  75;  Stop  =  5,  then  once  the  aircraft  has 
changed  a  total  of  75  or  more  feet  from  the  end  of  the  last  record,  a  new  “event”  will  be  initiated. 
This  new  event  will  end  when  the  aircraft  makes  a  change  in  altitude  of  5  feet  or  less  from  one 
sample  to  the  next.  (The  record  will  also  end  if  the  direction  of  movement  changes.) 

Table  1  below  lists  the  meaning  of  the  14  columns  contained  in  the  data  generator.  This  Table 
summarizes  the  contents  of  the  summary  generator.  The  output  file  is  formatted  for  input  into  a 
spreadsheet  or  into  another  program.  In  other  words,  using  this  output,  the  researcher  will  be 
able  to  extract  required  information  and  carry  out  statistical  analyses. 

TABLES.] 

CONTENTS  OF  THE  DATA  GENERATOR 


Column  TYPE  OF  DATA 


1  Record  Type 

0  =  Start/Stop 

1  =  Speed  change 

2  =  Altitude  change 

3  =  Heading  change 

4  =  Throttle  adjustment 

5  =  Pitch  deflection 

6  =  Roll  deflection 

7  =  Rudder  deflection 

8  =  Straight  &  Level 

9  =  Unusual  attitude 

10  =  User  input  (keypress) 

1 1  =  SA  measure 

2  Time  into  scenario  -  Elapsed  flying  time 

3  Duration  for  record  types  1  -9 

Raw  time  into  scenario  for  record  types  0,  10,  &  1 1 

4  Calibrated  airspeed  -  at  initiation  of  record 

5  Altitude  ■■  at  initiation  of  record 

6  Heading  -  at  initiation  of  record 

7  Position  North  -  at  initiation  of  record 
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8  Position  East  -  at  initiation  of  record 

9  Average  Angle-Of-Attack  (AOA)  for  record  type  2  (Altitude  changes) 

Average  Gz  pulled  for  record  type  3  (Heading  changes) 

Average  Deflection  value  for  records  5-7  (Flight  stick  and  rudder) 

Heading  deviation  for  record  type  8  (Straight  &  Level) 

10  Standard  Deviation  of  AOA  for  record  type  2  (Altitude  changes) 

Standard  Deviation  of  Gz  pulled  for  record  type  3  (Heading  changes) 

Standard  Deviation  of  Deflection  value  for  records  5-7  (Flight  stick  and  rudder) 
Altitude  Deviation  for  record  type  8  (Straight  &  Level) 

1 1  Max  Angle-Of-Attack  (AOA)  for  record  type  2  (Altitude  changes) 

Max  Gz  pulled  for  record  type  3  (Heading  changes) 

Max  Deflection  value  for  records  5-7  (Flight  stick  and  rudder) 

12  Direction  of  change  1  =  Up/Right/Increase 

- 1  =  Down/Left/Decrease 

13  Ending  Speed  for  record  types  1  &  4  (Speed/Throttle  changes) 

Ending  Altitude  for  record  types  2  &  5  (Altitude/Pitch  changes) 

Ending  Heading  for  record  types  3,  6  &  7  (Heading/Roll/Rudder  changes) 

SA  activity  for  record  type  1 1 

1 4  Speed  change  per  second  for  record  type  1 
Altitude  change  per  second  for  record  type  2 
Heading  change  per  second  for  record  type  3 
ASCII  value  of  keypress  for  record  type  10 
SA  message  for  record  type  1 1 


A  portion  of  an  actual  summary  output  is  shown  in  Table  8.2.  The  first  record  is  the  scenario 
start.  (Time  of  start,  0. 1 82.  is  the  time  of  the  first  aircraft  state  record.)  The  generator  then 
detected  two  rudder  movements,  first  to  the  right  then  to  the  left.  These  caused  the  heading  of 
the  aircraft  to  change  from  .01°  to  359.85°  then  from  359.78°  to  359.97°.  Then  a  speed  change 
is  detected  (throttle  motion  not  implemented  at  the  time  of  writing)  starting  at  1 1.142  seconds 
into  the  scenario  and  lasting  for  12.025  seconds.  The  speed  started  at  23.5  knots  and  ended  at 
261 .64  knots.  Another  rudder  movement  to  the  right  was  detected,  followed  by  an  upward  pitch 
deflection.  At  17.142  seconds  into  the  scenario  a  change  in  altitude  was  detected  resulting  in  a 
climb  for  6.025  seconds  and  an  altitude  change  from  3.41’  to  1 88.52’.  This  was  accompanied  by 
a  pitch  deflection  during  the  same  time  period.  The  pitch  deflection  is  down  and  it  appears  that 
the  subject  was  trying  to  reduce  the  natural  ascent  rate  accompanying  lift-off.  At  1 8.807  seconds 
into  the  scenario  (43.667  seconds  since  this  session  began  -  raw  time)  SA  message  number  7301 
was  displayed  resulting  in  a  response  by  the  subject  (ASCII  value  73  resulting  from  a  ‘Slider 
Down’  on  the  flight  stick)  at  raw  time  46.649.  It  is  possible  to  calculate  the  response  time  fcr  the 
message  to  be  2.982  seconds.  (The  next  record  is  a  result  of  the  ‘return  to  normal  position’  of  the 
slider  switch  on  the  flight  stick  and  may  be  disregarded  for  our  purposes.)  The  scenario  time 
clock  is  paused  while  SA  messages  are  displayed.  Two  more  pitch  deflections  are  then 
registered,  one  upward  then  one  downward.  A  user  input  of  ASCII  value  108  (in  our  scenario 
this  equates  to  raising  of  the  landing  gear  -  pressing  keyboard  ‘L’)  occurred  23.241  seconds  into 
the  scenario  when  the  aircraft  was  at  an  altitude  of  188.52’.  An  SA  message  7302  was  then 
displayed  at  scenario  time  25.072  seconds,  with  an  unusual  attitude  (SA  record  with  SA  activity 
2  and  indicating  unusual  attitude  1)  beginning  at  the  same  time. 
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TABLE  8.2 


SAMPLE  OF  OUTPUT  FROM  THE  DATA  GENERATOR 
(One  Subject  -  One  Segment  of  Flight) _ 


1 

2 

3 

0 

0.182 

0.000 

7 

0.182 

4.800 

7 

5.542 

3.600 

1 

11.142 

12.025 

7 

11.942 

2.240 

5 

16.182 

0.800 

2 

17.142 

6.025 

5 

17.382 

3.385 

11 

18.807 

43.667 

10 

18.807 

46.649 

10 

19.011 

46.886 

5 

20.767 

0.960 

5 

21.967 

1.600 

10 

23.241 

51.116 

11 

25.072 

52.947 

11 

25.072 

52.947 

10 

25.072 

55.067 

9 

25.072 

16.906 

10 

25.149 

55.173 

10 

42.474 

72.498 

11 

42.474 

72.498 

10 

43.316 

73.340 

11 

43.316 

73.340 

5 

43.978 

2.240  : 

6 

49.418 

0.800  1 

COLUMN 

8  9  10  1 


12  13  14 


2.59  9.89  0.01  1881 19.4  814786.0  0.00  0.00  0.00  0  0.00  1.00 
2.59  9.89  0.01188119.4  814786.0  0.14  0.03  0.17  I  359.85  0.00 
2.59  14.99  359.78  188240.7  814786.0  0.17  0.010.18-1  359.97  0.00 

2.59  23.50  0.30  188400.0  814786.0  0.00  0.00  0.00  1  261.64  20.94 
2.59  23.50  0.30  188400.0  814786.0  0.13  0.01  0.14  1  359.46  0.00 
2.70  125.34  358.64  188998.5  814782.3  0.12  0.01  0.13  1  2.70  0.00 

3.41  146.32  358.95  189218.1  814779.3  3.02  7.63  14.27  1  188.52  30.86 
3.41146.32  358.95  189218.1814779.3  0.610.210.92  -1  132.43  0.00 


22.10  162.86  359.01  189489.2  814774.4  0.00  0.00  0.00  0 

22.10  162.86  359.01  189489.2  814774.4  0.00  0.00  0.00  0 

22.10  162.86  359.01  189489.2  814774.4  0.00  0.00  0.00  0 

132.43  197.37  358.99  190072.1814762.1  0,25  0.13  0.35  I 
176.33  217.64  358.80  190433.9  814755.6  0.15  0.04  0.20-1 
188.52  261.64  358.97  191244.0  814743.1  0.00  0.00  0.00  0 
167.95  282.94  358.93  191685.3  814737.1 
113.01  319.07  292.19  192159.6  814729.3 
1 13.01  319.07  292.19  192159.6  814729.3 


0.00  7301.00 


0.00  0.00  0.00  0 
0.00  0.00  0.00  0 
0.00  0.00  0.00  0 
0.00  0.00  3.05  0 
0.00  0,00  0.00  0 
0.00  0.00  0.00  0 
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The  response  from  the  subject  (ASCII  value  73  -  ‘Slider  Down”)  to  the  SA  message  took  2.12 
seconds.  The  unusual  attitude  record  reports  that  it  took  1 6.906  seconds  for  the  subject  to 
recover  from  the  unusual  attitude.  The  SA  record  showing  the  start  of  the  unusual  attitude 
reports  the  aircraft  state  values  at  initiation  and  the  unusual  attitude  record  shows  the  ending  state 
values.  An  unusual  attitude  is  considered  to  be  over  when  the  wings  of  the  aircraft  are  level  for 
two  state  readings.  (The  next  record  is  another  ‘retum-to-normal’  record.)  At  scenario  time 
42.474  seconds  the  subject  input  a  waypoint  selection  (‘W’  on  the  keyboard)  and  the  next  SA 
record  reports  WPO  selected  (SA  activity  7  and  message  0.)  A  waypoint  selection  of  WPl 
follow.  The  sample  portion  ends  with  a  detected  upward  pitch  and  left  roll  deflection. 
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APPENDIX  9 


FLIGHT  PERFORMANCE  VALUES  OBTAINED 
IN  THE  DEMONSTRATION  PROJECT 


NORTH  DAKOTA  DEMONSTRATION  ANALYSIS  -  Mean  Performance  By 
Experience  Level  and  Attempt  Number 
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DEVIATION  (DEG.)  DEVIATION  (FT.) 


APPENDIX  10 


DATA  PLOTS  FOR  DEPENDENT  VARIABLES  THAT 
DID  NOT  DIFFER  SIGNIFICANTLY 


ALTITUDE  DEVIATION  -  STRAIGHT  AND  LEVEL  FLIGHT 
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HEADING  DEVIATION  -  STRAIGHT  AND  LEVEL  FLIGHT 
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DEVIATION  (FT.)  DEVIATION  (FT.) 


DEVIATION  FROM  ASSIGNED  ALTITUDE  FOR  ROLLOUT  ON 
GROUND  ATTACK  MISSION 


ATTEMPT  NUMBER 


DEVIATION  FROM  ASSIGNED  ALTITUDE  FOR  INITIATING  LEFT 
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SPEED  DEVIATION 


DEVIATION  FROM  ASSIGNED  SPEED  AT  SELECTION  OF  MIL- 

POWER 


-♦—LOW  EXP.  ;| 
. B . HIGH  EXP,; I 


d’  SA  MEASURES  FOR  GROUND  ATACK  MISSIONS  -  LOW 
EXPERIENCE  SUBJECTS 


SA  MEASURE  TYPE 


!—♦— TRIALS  1-10  I 
j  .-B . TRIALS  11-20  ; 


131 


SA  MEASURES  FOR  GROUND  ATTACK  MISSIONS  -  HIGH 
EXPERIENCE  SUBJECTS 


HIGH  "A”  HIGH  "B"  HIGH  "C" 

SA  MEASURE  TYPE 


d’  SA  MEASURES  FOR  GROUND  ATTACK  MISSION  BY 
EXPERIENCE  LEVEL 
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d'  SA  MEASURES  FOR  S-TURNS  BY  EXPERIENCE  LEVEL 
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SA  QUESTIONS  PERCENT  CORRECT  FOR  GROUND  ATTACK 
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SA  QUESTIONS  PERCENT  CORRECT  FOR  GROUND  ATTACK 
MISSIONS  -  HIGH  EXPERIENCE  SUBJECTS 
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SA  QUESTIONS  PERCENT 
CORRECT  FOR  GROUND  ATTACK 
MISSIONS  BY  EXPERIENCE  LEVEL 
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SA  QUESTIONS  PERCENT  CORRECT  FOR  S-TURNS  BY 
EXPERIENCE  LEVEL 
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SBIR/STTR  Awards  :  Small  Business....gy  Transfer :  Program  Awards  :  DoD 


B  ASIC  I 
Search  I 


DoD  SBIR/STTR 

BETAiUS  -  Ammm  Smatm  Rmmm 


Program:  SBIR 

Agency:  AF  Field  Office:  AL 

TOPIC  Number:  AF94-032  Control  Number:  94AL  -338 

Contract  Number:  F41624-95-C-5008  Phase:  2 

Awarded  In:  95  Award  Amount:  $67 1 ,892 

Award  Start  Date:  07MAR95  Award  Completion  Date:  07MAR97 

Proposal  Title:  Development  of  the  Low-Cost  Situation  Awareness  Flight  Trainer  Evaluator  (SAFTE) 

Principal  Investigator  Name:  Robert  D.  O'Donnell,  Ph.D 
Principal  Investigator  Phone:  513-254-3171 


Firm 

NTI,  INC. 

4130  Linden  Ave 
Dayton,  OH  45432 

Wompn  Owned:  N 
Minority  Owned:  N 
Number  of  Employees:  7 


Keywords:  SITUATION  AWARENESS  FLIGHT  TRAINING  SYNTHETIC  TASK  FLIGHT 
SIMULATOR  SITUATION  AWARENESS  EVALUATION 


